How to Calculate an Operation's Memory Consumption

 

How can you determine how much memory is consumed by a specific operation of a Unix program? Valgrind's Massif subsystem could help you in this regard, but it can be difficult to isolate a specific operation from Massif's output. Here is another, simpler way.

  1. Determine the process id of the program you want to check, using ps or top.
  2. Start recording the program's malloc and free operations using ltrace, and save the output to a file.
    ltrace -p 20907 -e malloc,free -o file.ltrace
    
  3. Perform the operation you want to measure.
  4. Terminate ltrace by pressing Ctrl-C.
  5. Run the following Perl script (you can download through this link) giving it as an argument the file that ltrace produced.
    #!/usr/bin/perl
    #
    # Read ltrace(1) output to calculate peak memory consumption
    # Diomidis Spinellis, September 2012
    
    use strict;
    use warnings;
    
    # Allocation overhead
    my $overhead = 16;
    
    my $current = 0;
    my $allocated = 0;
    my $freed = 0;
    my $peak = 0;
    
    my $nmalloc = 0;
    my $nfree = 0;
    
    # Size of allcoated memory blocks
    my %size;
    
    # Lines are of the form
    # 6312 malloc(16)                                  = 0x01979da0
    # 6312 free(0x0197ac50)                            = <void>
    # 6312 realloc(0x12b229c60, 64)                   = 0x136447130
    
    while (<>) {
    	if (/\d+ malloc\((\d+)\)\s+\=\s*0x(\w+)/) {
    		$size{$2} = $1;
    		$current += $1 + $overhead;
    		$allocated += $1 + $overhead;
    		$nmalloc++;
    		$peak = $current if ($current > $peak);
    	} elsif (/\d+ free\(0x(\w+)\)/) {
    		my $block_size = (defined($size{$1}) ? $size{$1} : 0) + $overhead;
    		$current -= $block_size;
    		$freed += $block_size;
    		$nfree++;
    	} elsif (/\d+ realloc\(0x(\w+), (\d+)\)\s+\=\s*0x(\w+)/) {
    		my $block_size = (defined($size{$1}) ? $size{$1} : 0) + $overhead;
    		$current -= $block_size;
    		$freed += $block_size;
    		$nfree++;
    		$size{$3} = $2;
    		$current += $2 + $overhead;
    		$allocated += $2 + $overhead;
    		$nmalloc++;
    		$peak = $current if ($current > $peak);
    	} else {
    		print STDERR "Unmatched line: $_";
    	}
    }
    
    print qq{Allocated: $allocated bytes in $nmalloc calls
    Freed: $freed bytes in $nfree calls
    Peak marginal memory use: $peak bytes
    };
    
The result will be a report, like the following.
$ perl calcmem.pl file.ltrace
Allocated: 15155 bytes in 332 calls
Freed: 10341 bytes in 218 calls
Peak marginal memory use: 9715 bytes

Comments   Toot! Share


Last modified: Saturday, September 22, 2012 5:46 pm

Creative Commons Licence BY NC

Unless otherwise expressly stated, all original material on this page created by Diomidis Spinellis is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.