GUI for HdbExtractor - TANGO Controls

# 9 years ago
giovanni	Hi all. Today we installed Hdb++ successfully for the first time, using the Configuration Manager by hand to add our attributes to the Event Subscriber. The system seems to work, and now we're trying to use any of the Qt GUI that there are in the repository to extract the data in the Hdb database. However, we're not able to find out which is the tools that I need to compile, and then how to make them work. May you help us? Giovanni

# 9 years ago

Hi all. Today we installed Hdb++ successfully for the first time, using the Configuration Manager by hand to add our attributes to the Event Subscriber. The system seems to work, and now we're trying to use any of the Qt GUI that there are in the repository to extract the data in the Hdb database. However, we're not able to find out which is the tools that I need to compile, and then how to make them work.

May you help us?

Giovanni

# 9 years ago
sergi_rubio	I think the current viewer for HDB++ is eGiga (web viewer, php based). But, if a PyQt GUI is ok and if you use MySQL and the old-schema (1 table/attribute named like att_00001) you can use taurus to plot/extract values from archiving. Get PyTango and Taurus: https://pypi.python.org/pypi/PyTango https://pypi.python.org/pypi/taurus Get PyTangoArchiving: https://sourceforge.net/p/tango-cs/code/HEAD/tree/archiving/tool/PyTangoArchiving PyTangoArchiving Recipes: http://www.tango-controls.org/resources/howto/pytangoarchiving/ Define your HdbExtractor.DbConfig class property: DbConfig: user:password@yourmysqlhost Put both in your PYTHONPATH and then use the ArchivingBrowser to locate/plot/export the attributes you want: python PyTangoArchiving/widget/ArchivingBrowser.py Hope it helps, Sergi Rubio Keep on dancing, http://www.tango-controls.org/resources/howto/how-fandango/ Edited 9 years ago

# 9 years ago

sergi_rubio

I think the current viewer for HDB++ is eGiga (web viewer, php based).

But, if a PyQt GUI is ok and if you use MySQL and the old-schema (1 table/attribute named like att_00001) you can use taurus to plot/extract values from archiving.

Get PyTango and Taurus:

https://pypi.python.org/pypi/PyTango
https://pypi.python.org/pypi/taurus

Get PyTangoArchiving:

https://sourceforge.net/p/tango-cs/code/HEAD/tree/archiving/tool/PyTangoArchiving

PyTangoArchiving Recipes:

http://www.tango-controls.org/resources/howto/pytangoarchiving/

Define your HdbExtractor.DbConfig class property:

DbConfig: user:password@yourmysqlhost

Put both in your PYTHONPATH and then use the ArchivingBrowser to locate/plot/export the attributes you want:

python PyTangoArchiving/widget/ArchivingBrowser.py

Hope it helps,

Sergi Rubio

Keep on dancing,

http://www.tango-controls.org/resources/howto/how-fandango/

Edited 9 years ago

# 9 years ago
giovanni	Thanks Sergi. I'm trying E-Giga, but it does not seem to work witk HDB++. I've downloaded the last version (1.6.8), configured it, but the variable list is empty in the log.php page. Are you sure it supports HDB++ too? In the afternoon I'll try PyTangoArchiving too. Now I'm not able to install PyQwt5 on Scientific Linux (no packages found, I'm investigating), and so I cannot run the tool. Regards, Giovanni

# 9 years ago

giovanni

Thanks Sergi. I'm trying E-Giga, but it does not seem to work witk HDB++. I've downloaded the last version (1.6.8), configured it, but the variable list is empty in the log.php page. Are you sure it supports HDB++ too?

In the afternoon I'll try PyTangoArchiving too. Now I'm not able to install PyQwt5 on Scientific Linux (no packages found, I'm investigating), and so I cannot run the tool.

Regards,

Giovanni

# 9 years ago
Reynald	For HDB++, there is an hdbextractor GUI developed in C++/Qt by Elettra (source code available on sourceforge). The Cassandra C++ extraction part still needs to be developed. At the ESRF, we also have a prototype Java GUI to extract data from HDB++ MySQL and HDB++ Cassandra. Please note that HDB++ is still in development phase so these tools will be improved and other tools might be developed to extract data. They are not yet officially supported. There is a plan to make HDB++ compatible with Mambo too for the extraction part. Rosenberg's Law: Software is easy to make, except when you want it to do something new. Corollary: The only software that's worth making is software that does something new.

# 9 years ago

Reynald

For HDB++, there is an hdbextractor GUI developed in C++/Qt by Elettra (source code available on sourceforge).
The Cassandra C++ extraction part still needs to be developed.
At the ESRF, we also have a prototype Java GUI to extract data from HDB++ MySQL and HDB++ Cassandra.
Please note that HDB++ is still in development phase so these tools will be improved and other tools might be developed to extract data. They are not yet officially supported.
There is a plan to make HDB++ compatible with Mambo too for the extraction part.

Rosenberg's Law: Software is easy to make, except when you want it to do something new.
Corollary: The only software that's worth making is software that does something new.

# 9 years ago
giovanni	Thanks Reynald. I've tried the PyTangoArchiving just few moments ago. I can see all the attributes of all the servers, but I'm not able to configure the archiver and the HDB++ database (server, user, password, name, …). I don't understand if it is a problem of the configuration of just a limitation of the tool that supports only the old HDB. The C++/Qt hdbextractor will be tested in the next days. Then, another question: we are using only the Event Subscriber and the Configuration Manager for Hdb++. In order to use the extracting tools, I understand that I need also a HdbExtractor device running. Is it the Java-coded one? Do I need also the HdbArchiver? We are not using it at the moment. I'm sorry, we are a little bit confused… Can you explain us which is the best deployment to use the Hdb++ system? Nevertheless, I've a last question for you. We are still in hard development, and in these days we are just trying to use the HDB system to see what we can do with it. We need to monitor several hundreds of Tango devices, and about a thousand of attributes. So far we've found HDB++ working pretty well, ad least for what concern the archiving system. Being HDB++ still in development, do you suggest to use the old HDB instead of HDB++? We need to decide as soon as possible the architecture of the archiving system. Thanks for the support. Giovanni Edited 9 years ago

# 9 years ago

giovanni

Thanks Reynald. I've tried the PyTangoArchiving just few moments ago. I can see all the attributes of all the servers, but I'm not able to configure the archiver and the HDB++ database (server, user, password, name, …). I don't understand if it is a problem of the configuration of just a limitation of the tool that supports only the old HDB.

The C++/Qt hdbextractor will be tested in the next days.

Then, another question: we are using only the Event Subscriber and the Configuration Manager for Hdb++. In order to use the extracting tools, I understand that I need also a HdbExtractor device running. Is it the Java-coded one? Do I need also the HdbArchiver? We are not using it at the moment. I'm sorry, we are a little bit confused… Can you explain us which is the best deployment to use the Hdb++ system?

Nevertheless, I've a last question for you. We are still in hard development, and in these days we are just trying to use the HDB system to see what we can do with it. We need to monitor several hundreds of Tango devices, and about a thousand of attributes. So far we've found HDB++ working pretty well, ad least for what concern the archiving system. Being HDB++ still in development, do you suggest to use the old HDB instead of HDB++? We need to decide as soon as possible the architecture of the archiving system.

Thanks for the support.

Giovanni

Edited 9 years ago

# 9 years ago
Reynald	Hi Giovanni, As far as I know PyTangoArchiving is not supporting HDB++ for the moment. There is a java GUI to help you to configure the HDB++ archiving. (svn://svn.code.sf.net/p/tango-cs/code/archiving/hdb++/gui/java-configurator) You can also use the Configuration Manager interface directly by code if you prefer. In HDB++, there is no HdbExtractor device which could be a bottleneck. The extraction GUIs will talk directly to the database, via an abstraction layer to allow to talk to MySQL or to Cassandra (or eventually something else in the future). HDB++ is still in development and has not been officially released, even though it is already archiving operational data (in parallel of our previous HDB system) at the ESRF (HDB++ MySQL and Cassandra) and in Elettra (HDB++ MySQL). What I meant when writing "it is still in development" is that it will still evolve, some features are still missing and the documentation is quite poor for the moment. You might get very limited support too during the development phase since it is not officially released. When do you plan to put your archiving system in operation? Rosenberg's Law: Software is easy to make, except when you want it to do something new. Corollary: The only software that's worth making is software that does something new.

# 9 years ago

Reynald

Hi Giovanni,

As far as I know PyTangoArchiving is not supporting HDB++ for the moment.
There is a java GUI to help you to configure the HDB++ archiving. (svn://svn.code.sf.net/p/tango-cs/code/archiving/hdb++/gui/java-configurator)
You can also use the Configuration Manager interface directly by code if you prefer.

In HDB++, there is no HdbExtractor device which could be a bottleneck. The extraction GUIs will talk directly to the database, via an abstraction layer to allow to talk to MySQL or to Cassandra (or eventually something else in the future).

HDB++ is still in development and has not been officially released, even though it is already archiving operational data (in parallel of our previous HDB system) at the ESRF (HDB++ MySQL and Cassandra) and in Elettra (HDB++ MySQL).

What I meant when writing "it is still in development" is that it will still evolve, some features are still missing and the documentation is quite poor for the moment. You might get very limited support too during the development phase since it is not officially released.

When do you plan to put your archiving system in operation?

Rosenberg's Law: Software is easy to make, except when you want it to do something new.
Corollary: The only software that's worth making is software that does something new.

# 9 years ago
Reynald	As Sergi wrote, if you are using HDB++ with the old HDB MySQL schema, you might be able to use some tools already developed for HDB to extract and visualize the data. I don't think you can use the old tools to configure your archiving though. I think he is the best to help you in this matter. Rosenberg's Law: Software is easy to make, except when you want it to do something new. Corollary: The only software that's worth making is software that does something new.

# 9 years ago

Reynald

As Sergi wrote, if you are using HDB++ with the old HDB MySQL schema, you might be able to use some tools already developed for HDB to extract and visualize the data. I don't think you can use the old tools to configure your archiving though.
I think he is the best to help you in this matter. smile

Rosenberg's Law: Software is easy to make, except when you want it to do something new.
Corollary: The only software that's worth making is software that does something new.

# 9 years ago
giovanni	Hi Reynald. I'll try to explain our situation as better as I can. The system that we are developing is the System Supervisor for Advanced Virgo, that is going to be operative in the next few months. The Tango devices of the devices that we need to control are more or less working, and now we need to add some tools to them, as for example a reliable archiving system. We like Hdb++ because it works with an independent database, independent from Tango, and does not need any Tango devices to extract the stored data. We've put it in a clustered MySQL database (ndbcluster as engine), and seems to work very well. [If it can be interesting, also the Tango database is running in the same server with the same engine. There are only few bugs/incompatibilities between it and InnoDB/MyISAM in the Databased server with NULL values passed to a NOT NULL column, that can be easily fixed. But this is another story.] On the other hand the old HDB system, that I've found in the ArchivingRoot package, even if well supported, seems to need a more complex infrastructure, as for example some privileges to the MySQL user that we cannot have in this clustered database. I don't know how much I've been clear, however this is more or less why we prefer to use the Hdb++ database. Accidentally, unaware of these facts, we have already installed the new schema, and for now we've decided to try it a little bit more before a possible switch to the old schema. I'm trying to compile the QHdbExtractor found in the svn within the Hdb++ folder, under the archiving/hdb++/hdbextractor/cpp/tags/release-0.90.0 folder. Is it the GUI developed in C++/Qt by Elettra that you mentioned some posts ago? Giovanni Edited 9 years ago

# 9 years ago

giovanni

Hi Reynald. I'll try to explain our situation as better as I can.

The system that we are developing is the System Supervisor for Advanced Virgo, that is going to be operative in the next few months. The Tango devices of the devices that we need to control are more or less working, and now we need to add some tools to them, as for example a reliable archiving system. We like Hdb++ because it works with an independent database, independent from Tango, and does not need any Tango devices to extract the stored data. We've put it in a clustered MySQL database (ndbcluster as engine), and seems to work very well. [If it can be interesting, also the Tango database is running in the same server with the same engine. There are only few bugs/incompatibilities between it and InnoDB/MyISAM in the Databased server with NULL values passed to a NOT NULL column, that can be easily fixed. But this is another story.]

On the other hand the old HDB system, that I've found in the ArchivingRoot package, even if well supported, seems to need a more complex infrastructure, as for example some privileges to the MySQL user that we cannot have in this clustered database.

I don't know how much I've been clear, however this is more or less why we prefer to use the Hdb++ database. Accidentally, unaware of these facts, we have already installed the new schema, and for now we've decided to try it a little bit more before a possible switch to the old schema.

I'm trying to compile the QHdbExtractor found in the svn within the Hdb++ folder, under the archiving/hdb++/hdbextractor/cpp/tags/release-0.90.0 folder. Is it the GUI developed in C++/Qt by Elettra that you mentioned some posts ago?

Giovanni

Edited 9 years ago

# 9 years ago
sergi_rubio	Hi Reynald, Giovanni Well if you are already using the new MySQL schema (1 table/type instead of 1 table/attr) then PyTangoArchiving is not an option "yet"; I plan to add support for the new schema and cassandra after summer so probably the work of installing taurus/qwt will be still profitable. If you are using the new HDB++ with old schema (tables like att_00001) then it should work well if you configured the HDBExtractor.DbConfig class property. You just need to set the property,the device itself is not needed because PyTangoArchiving "bypasses" the HdbExtractor and queries MySQL directly. Unfortunately, you'll still need the device servers to configure archiving from python tools/scripts. Integrating the HDB++ configuration part in PyTangoArchiving will take longer (2016) than just extraction. Good luck! Sergi Keep on dancing, http://www.tango-controls.org/resources/howto/how-fandango/

# 9 years ago

sergi_rubio

Hi Reynald, Giovanni

Well if you are already using the new MySQL schema (1 table/type instead of 1 table/attr) then PyTangoArchiving is not an option "yet"; I plan to add support for the new schema and cassandra after summer so probably the work of installing taurus/qwt will be still profitable.

If you are using the new HDB++ with old schema (tables like att_00001) then it should work well if you configured the HDBExtractor.DbConfig class property. You just need to set the property,the device itself is not needed because PyTangoArchiving "bypasses" the HdbExtractor and queries MySQL directly.

Unfortunately, you'll still need the device servers to configure archiving from python tools/scripts. Integrating the HDB++ configuration part in PyTangoArchiving will take longer (2016) than just extraction.

Good luck!

Sergi

Keep on dancing,

http://www.tango-controls.org/resources/howto/how-fandango/

# 9 years ago
sergi_rubio	Anybody knows if eGiga will be adapted to the new HDB++ schema? Sergi Keep on dancing, http://www.tango-controls.org/resources/howto/how-fandango/