Memory leak
avatar par défaut
AlexS
Hello,

I have a test server which generates signal (a sum of two sin functions and noise) as spectrum of doubles (10000 points). I use two buffers one for signal (it is calculated in write_attr_hardware)
and output buffer. These buffers are allocated in init_device() and freed in delete_device().
As follows:

void MyVectorGenerator::init_device()
{
  ………
   /*—– PROTECTED REGION END —–*/    //    MyVectorGenerator::init_device_before
    number = 10;
    buffer_size = number;
    pre_calc_buffer = (double*)malloc(buffer_size*sizeof(double));
    out_buffer = (double*)malloc(buffer_size*sizeof(double));
    set_state(Tango::OFF);
    set_status("Generator is OFF");
    /*—– PROTECTED REGION END —–*/    //    MyVectorGenerator::init_device
}
…..
void MyVectorGenerator::delete_device()
{
    DEBUG_STREAM << "MyVectorGenerator::delete_device() " << device_name << endl;
    /*—– PROTECTED REGION ID(MyVectorGenerator::delete_device) ENABLED START —–*/

    //    Delete device allocated objects
    free(pre_calc_buffer);
    free(out_buffer);
    /*—– PROTECTED REGION END —–*/    //    MyVectorGenerator::delete_device
    delete[] attr_signal_read;
}
….
void MyVectorGenerator::write_attr_hardware(TANGO_UNUSED(vector<long> &attr_list))
{
    for(int i=0;i<number;i++)
    {
        //pre_calc_buffer[i] = amplitude1*sin(freq1*i)+amplitude2*sin(freq2*i);
        pre_calc_buffer[i] = amplitude1*sin(freq1*i)+amplitude2*sin(freq2*i)+noise_scale*noise_ampl*( (rand()%100) /100.);
    }
}
….
void MyVectorGenerator::read_signal(Tango::Attribute &attr)
{
    attr.set_value(attr_signal_read, 20000);
    if(state()==Tango::ON)
    {
        int rand_shift = rand()/number;
        for(int i=0;i<number;i++)
        {
            out_buffer[i] = pre_calc_buffer[(i+rand_shift)%number];
        }
        attr.set_value(out_buffer,number,0,false);
    }
}

Memory consumption grows 1MB per 10 minutes, when client polling.
Unfortunately valgrind doesn't work on out powerpc configuration.

Some exercises with __malloc_hook and __free_hook were done and suspicious function was found:
Tango::PollRing::force_copy_data(Tango::AttributeValueList_4*)
Configuration:
CPU powerpc
OS debian lenny
gcc 4.3.2
Tango 8.1.2
zeromq 4.1.2
omniorb 4.1.6

On x86_64 (kernel 3.13 Ubuntu( Mint), gcc 4.8.2 ) it runs without memory issue.

Do you have any suggestion how can I solve this problem?
rbourtembourg avatar
Reynald
Hi Alexs,

I will try to help you with my modest experience…
Is your signal attribute polled on the server side too?
What is the polling frequency on the server side if any?
If there is any polling on the server side, what is the value of your poll ring depth?
What is your client polling frequency?

If I were you, I would try to make only one call to 
attr.set_value()
in 
MyVectorGenerator::read_signal(Tango::Attribute &attr)
method.

In the code you pasted, if the state is Tango::ON, 
attr.set_value()
is called twice in your method.

I would suggest to try the following code instead:
void MyVectorGenerator::read_signal(Tango::Attribute &attr)
{
    if(state()==Tango::ON)
    {
        int rand_shift = rand()/number;
        for(int i=0;i<number;i++)
        {
            out_buffer[i] = pre_calc_buffer[(i+rand_shift)%number];
        }
        attr.set_value(out_buffer,number,0,false);
    }
    else
    {
        attr.set_value(attr_signal_read, 20000);
    }
}

Rosenberg's Law: Software is easy to make, except when you want it to do something new.
Corollary: The only software that's worth making is software that does something new.
avatar par défaut
AlexS
Thank you Reynald.

Yes, it seems that multiple calls of set_value could lead to memory leak.
But suggested changes did not affect memory usage growth.

After some investigation, it was found that when server increases memory consumption, taurus client issues a warning:
Dummy-2 WARNING 2015-07-30 18:24:55,560 devhost:10000.simple2/gen/dev1.signal.configuration:
"Zombie" object (TangoConfiguration(devhost:10000/simple2/gen/dev1/signal?configuration)) received an event. Unsubscribing it.

Taurus client is quite simple. It periodically prints attribute value.


………
def print_signal(devices):

    for dev in devices:
        print dev.getSimpleName()#signal.name
        try:
            signal=dev.getAttribute("signal")
            value =  signal.read(False)
            print dev.getSimpleName(),signal.name, value.value[0:5]

        except PyTango.ConnectionFailed as e:
          log(dev.getSimpleName(),signal.getSimpleName(),e,echo=True)
          raise
        except Exception as e:
            print str(e)
            print "Exception, skipping…."
…….
def periodical_read(devices,period=10):

    while True:
        print_signal(devices)
        time.sleep(period)
def main():
    #setting up devices
    ………
    periodical_read(devices,1)


When i run jive as client, memory usage is a constant.

It seems to be a interaction problem.

Could you suggest the direction of further investigation.
 
Register or login to create to post a reply.