PyTango 9.4.x - 9.5.x memory leak

# 2 weeks ago
Adriánmm89	Hi all, First of all, my system is using: Python 3.11 PyTango 9.4.2 cppTango 9.5.0 omniORB 4.3.0 zmq 4.3.2 tangoidl 5.1.0 If a device server is running, the memory usage keeps increasing. Debugging the system, I have come to the conclusion that the bug comes from PyTango. I have tried to solve it by installing the 9.5.x version (even maintenance/9.5.x "https://gitlab.com/tango-controls/pytango/-/tree/maintenance/9.5.x?ref_type=heads", generating myself the .whl package for Linux aarch64), but the only one that works properly is the PyTango 10.0.0 version. My problem is that I have to install 9.5.X versions to maintain compatibility with the rest of the systems in my infrastructure. Is there any way to solve this memory leak using a 9.5.X version? Is there any package already generated with a 9.5.x version without this bug for aarch64-Linux and Python3.11? Thanks in advance, Adrián.

# 2 weeks ago

Hi all,

First of all, my system is using:

Python 3.11
PyTango 9.4.2
cppTango 9.5.0
omniORB 4.3.0
zmq 4.3.2
tangoidl 5.1.0

If a device server is running, the memory usage keeps increasing.

Debugging the system, I have come to the conclusion that the bug comes from PyTango.

I have tried to solve it by installing the 9.5.x version (even maintenance/9.5.x "https://gitlab.com/tango-controls/pytango/-/tree/maintenance/9.5.x?ref_type=heads", generating myself the .whl package for Linux aarch64), but the only one that works properly is the PyTango 10.0.0 version.

My problem is that I have to install 9.5.X versions to maintain compatibility with the rest of the systems in my infrastructure.

Is there any way to solve this memory leak using a 9.5.X version?

Is there any package already generated with a 9.5.x version without this bug for aarch64-Linux and Python3.11?

Thanks in advance, Adrián.

# 2 weeks ago
Anton	Hi Adrián Sorry about the leaks! You are right that there are memory leaks in PyTango 9.4.x and 9.5.x. Some leaks are from PyTango and some are from cppTango (which PyTango wraps). The issues we can fix in PyTango are fixed in the 9.5.x maintenance branch you tried. However, there are still some issues in cppTango 9.5.0. Since the 9.5.x branch of cppTango is no longer getting updates, we can't fix it in PyTango either. In this PyTango MR, https://gitlab.com/tango-controls/pytango/-/merge_requests/750, a memory leak test was added, and it shows problems with: - reading r/w string attributes when the quality is AttrQuality.ATTR_INVALID - pushing events, when there is no subscriber - pushing events, when there is a subscriber In PyTango 10.0.0 (which wraps cppTango 10.0.x), this same test passes, so those memory leaks are fixed. The only known leak that remains is https://gitlab.com/tango-controls/cppTango/-/issues/368. I'm not sure what you mean about maintaining compatibility with the rest of the systems in your infrastructure. You can mix Tango clients and device servers running different Tango versions. The PyTango API has changed slightly, since 9.5.x, in which case you might need some changes to existing Python code. Regards, /Anton

# 2 weeks ago

Anton

Hi Adrián

Sorry about the leaks! You are right that there are memory leaks in PyTango 9.4.x and 9.5.x. Some leaks are from PyTango and some are from cppTango (which PyTango wraps).

The issues we can fix in PyTango are fixed in the 9.5.x maintenance branch you tried. However, there are still some issues in cppTango 9.5.0. Since the 9.5.x branch of cppTango is no longer getting updates, we can't fix it in PyTango either.

In this PyTango MR, https://gitlab.com/tango-controls/pytango/-/merge_requests/750, a memory leak test was added, and it shows problems with:
- reading r/w string attributes when the quality is AttrQuality.ATTR_INVALID
- pushing events, when there is no subscriber
- pushing events, when there is a subscriber

In PyTango 10.0.0 (which wraps cppTango 10.0.x), this same test passes, so those memory leaks are fixed. The only known leak that remains is https://gitlab.com/tango-controls/cppTango/-/issues/368.

I'm not sure what you mean about maintaining compatibility with the rest of the systems in your infrastructure. You can mix Tango clients and device servers running different Tango versions. The PyTango API has changed slightly, since 9.5.x, in which case you might need some changes to existing Python code.

Regards,
/Anton

# 1 week ago
t-b	Yes, cppTango developer here, and the leaks are indeed awful and should not be there. There is a slight change that the leak you found is not on our radar yet. So are you able to come up with a minimal reproducer?