I would like the reader to excuse my poor english.

Mark_malloc tutorial

This tutorial is related to mark_malloc 1.0.2.

Supported platforms

Mark_malloc is designed for unix systems, with binaries conforming to the System V Application Binary Interface (SYSV ABI) (I take the SYSV ABI as a reference to watch the stack for the backtrace). Go here for more information about the SYSV ABI. Search the web for specific ABI (each architecture defines its own rules, for example there is an ABI complement for sparc or intel ia32 architectures), "system V application binary interface" is a good searching phrase.

See here for supported systems.

Traced functions

The functions we trace are :
void *calloc(size_t, size_t);
void *malloc(size_t);
void free(void *);
void *realloc(void *, size_t);
Every call to any one of those functions will be redirected to our own, so we can check the memory usage of the traced program.

If you use other functions, mark_malloc will be absolutly useless.

Environment variables

Here are the environment variables used by mark_malloc.
All the examples here are related to bash.

Download

Go to http://sed.free.fr/mark_malloc and get the software in the download section.

Build

Untar the archive (tar -xzf or gunzip then tar -xf).

There is a Makefile for each system.

First of all, you need binutils in order to get the backtrace in a human readable fashion. So before everything, binutils must be installed on your system. Go get it. (I used the 2.11.2 version to develop mark_malloc, maybe you will have troubles if you use another version, just try and tell me.)

Unfortunately, and this is a big problem, our beloved binutils coders don't provide a shared version of libiberty, which we need !

This post explains the reasons. It is said that different projects use different versions of libiberty and if there was a libiberty.so on your machine, conflicts would appear. One might wonder why all those projects can't agree upon a common usage of libiberty.

So, here is how to build the libiberty.so for some supported platform.

For sparc/solaris, here is a page discribing how to do it.
I repeat here the essential (you are supposed to be in the libiberty subdirectory of the binutils package).

You could also generate libiberty.so by configuring with --enable-shared
and then generate the shared library using: 

/usr/ccs/bin/ld -G -Bdynamic pic/*.o -o libiberty.so 

Either copy libiberty.so to the directory where you wish to install it or use 

make libdir=/my/install/dir install 

For ia32/linux (iX86, what we wrongly call "pc"), here is the procedure to follow.

The binutils should now be perfectly well installed.

For other systems, try one of the previous methods, it should work. If not, contact me, I may help you.

Now, you will have to configure in your Makefile BFD_INC_DIR and BFD_LIB_DIR, which must point to the place where (respectively) the include and lib directories of binutils are stored on your system. (If they are installed in defaults directories, you don't need to set those variables, so you can comment them with a leading '#'. libiberty.so must be accessible to your compiler too, so maybe you will have to change the Makefile a bit, to add a -L<directory where libiberty.so is stored> somewhere.)

Now that you've done that, you can do a 'make -f Makefile.xxxx' where xxxx is your system. If you don't find a Makefile that corresponds to your system, it's probably because mark_malloc doesn't work on it. Contact me to solve this as soon as possible !

Now, you should have a libmark_malloc.so. If there is a problem, don't hesitate to contact me.

Install

There is no particular installation process.

You just must remember in which directory libmark_malloc.so is stored.

Using mark_malloc

Let's examine how to use mark_malloc through a little example.

As I am familiar with bash, we will work with this shell. If you prefer csh or the like, you will have to change some stuff here and there in the following (export becomes more or less setenv for instance).

Here is a little program, with a faulty malloc. It is absoltly useless, but serves as an illustration.

#include <stdio.h>
#include <stdlib.h>

void c(char *p)
{
  char *t = malloc(100);
  int i;

  for (i=0; i<100; i++)
    t[i] = p[i];
}

void b(void)
{
  char t[100];

  c(t);
}

int main(void)
{
  b();

  return 0;
}
Let's suppose libmark_malloc.so is in the current directory. So one would do :
/home/roux/jmi/mark_malloc/release>gcc -g -o test test.c
Here, you see we build the program with -g. You must use this flag, in order to get the backtrace information to the end of the program. And you can't use too much optimization for your code, because we need stack frames. For example, ia32 architecture code can be compiled with the flag -fomit-frame-pointer, which will result in a strange behavior, due to the way we inspect the stack on this architecture (unfortunately for us, the good way (the one we suppose in our backtrace routine) to set the stack on ia32 is specified as optional in the ABI, so you must compile your code for this option to be turned on, so don't optimize too much ; -O2 is cool). (I don't know if mpr behaves well in this situation. The stack inspection routine is very strange in mpr.)

Now, the program is ready. You must configure mark_malloc to work with it.

So, you will have to do the following.

/home/roux/jmi/mark_malloc/release>export MARK_MALLOC_LIBC_PATH=/usr/lib/libc.so
/home/roux/jmi/mark_malloc/release>export MARK_MALLOC_PROGRAM_FILE=./test
The first one is very important. It must point to the libc of your system (probably /lib/libc.so.6 under ia32/linux and maybe most of the gnu systems).

The second one points to the program you will debug. This is used to get the line number/file information when you will get the backtrace. Beware to let it point to the correct file ! Here, while developping mark_malloc with a teacher, we had a problem with that. We got strange line number/functions calls, we didn't understand what was going on. It was just a wrong MARK_MALLOC_PROGRAM_FILE.

And, of course, don't forget to set your LD_LIBRARY_PATH so it will point to libbfd and libiberty (which are the ones we use from the binutils package).

/home/roux/jmi/mark_malloc/release>export LD_LIBRARY_PATH=/home/roux/inge/asm/binutils/lib:$LD_LIBRARY_PATH 
Ok, now everything is ready to run the program. Let's see what it gives.
/home/roux/jmi/mark_malloc/release>LD_PRELOAD=./libmark_malloc.so ./test
mark_malloc warning : MARK_MALLOC_SAVE_TO environment variable not declared (stderr will be used)
nb_marked = 1 nb_malloc = 1 nb_free = 0 nb_malloc-nb_free = 1
number of unfreed buffer = 1, showing 1 traces
malloc error #0 trace :
??      ??      0
c       /home/roux/jmi/mark_malloc/release/test.c       6
b       /home/roux/jmi/mark_malloc/release/test.c       17
main    /home/roux/jmi/mark_malloc/release/test.c       22
_start  ??      0
-----------------------------------
/home/roux/jmi/mark_malloc/release>
Ouch ! Horror ! We've got an error. Damned ! we didn't expect it.

You can see the traces of the program (we've got only one error). _start called main which called b which called c which probably called malloc (yes, it did !). We don't see malloc but ??, I don't really know why. That's a libbfd problem (or maybe I don't use this library the right way, which is possible too).

Under ia32/linux, I always have one strange trace, looking like this :

??      ??      0
??      ??      0
??      ??      0
??      ??      0
_start  ??      0
-----------------------------------
I don't know what it is. Don't worry about it. As you can see, main does not appear in the trace, so the problem is not in your code.

Let's go back to our example. You can see a warning too. MARK_MALLOC_SAVE_TO is not set. This variable will point to a file where the traces will be saved to the end of the program, for it not to interfer with your own data printed to the terminal. You can specify "stdout" and "stderr" too, which will have their common meaning (ie. mark_malloc won't create a file named "stdout" if you did a "export MARK_MALLOC_SAVE_TO=stdout", but will write to stdout, as expected). As you can see here, if you don't specify anything, stderr will be used.

As you can see, we used the LD_PRELOAD facility to force solaris (this test was done on a sparc) preload (that's the word) our library, which redefines malloc, calloc, and realloc, so that when the traced program calls one of those functions, our will be called instead. (we redirect exit and _exit too, for printing the traces to the end of the program).

You can specify the maximum number of printed traces. Imagine (yes I know it's probably bad assumption, but just imagine for a moment) that your program generates thousands of malloc that are never freed. To the end of its run, you'll see thousands of traces, which might be too huge for you. So here is the MARK_MALLOC_TRACES_MAX variable, which has an obvious meaning.

And coming with version 1.0.2 of the lib, you now have the MARK_MALLOC_HEXDUMP_CONTENT variable, which, when declared with the value 1, let the library dump the content of the unfreed buffers.

Intrusive debugging

The previous example has shown a totally non-intrusive usage of mark_malloc. We build our program the same way we would have done if mark_malloc wouldn't have been used. We didn't change a line of code inside it either. This is fully non-intrusive.

One can use mark_malloc as an intrusive tool, for not having reports of well known and wanted leaks (for instance, at startup of program, lots of malloc could be done, which we know they are not a problem).

Thus, you'll need to link libmark_malloc.so to your program before to execute it. Use a -lmark_malloc with a optional leading -L<directory where libmark_malloc.so is stored> and don't forget to add this directory to LD_LIBRARY_PATH before to run the program.

The intrusive method give you access to those functions 

void init_mark_malloc(void);
void aam(void);
void bbm(void);
void mark_malloc(void *);
init_mark_malloc must be called as soon as possible, for example as the first statement into your main function.

aam and bbm are used to enable/disable the marking. If a malloc appears between an aam and a bbm, it will be traced, if not, it won't. If there is a function call between an aam and a bbm and if this function (or one of its children) calls malloc (or calloc or realloc), it will be traced too.
See aam and bbm as parenthesis.
Why "aam" and "bbm" ? It's short (three letters) and aam comes from "auto mark malloc" which I mispelled "aam". And bbm was chosen in a quick fashion. This was the thing that came into my mind. Well, well, well, that's the story.

mark_malloc takes as argument the pointer you just malloced. It will mark the buffer, so you can see if this particular malloc is buggy or not. You can use it between an aam and a bbm, but you'll get a warning from mark_malloc (mark_malloc: WARNING trying to mark an already marked buffer), but that's not a serious problem. If you've got this warning in others contexts, this is a problem and contact me in the millisecond !

You can inspect mark_malloc.c and add the convenient functions you think you need.

As an example, our previous program could now be :

#include <stdio.h>
#include <stdlib.h>

void c(char *p)
{
  char *t = malloc(100);
  int i;

  mark_malloc(t);

  for (i=0; i<100; i++)
    t[i] = p[i];
}

void b(void)
{
  char t[100];

  c(t);
}

int main(void)
{
  init_mark_malloc();

  b();

  return 0;
}
Or with aam/bbm :
#include <stdio.h>
#include <stdlib.h>

void c(char *p)
{
  char *t = malloc(100);
  int i;

  /* no mark_malloc needed here, aam makes it implicit */

  for (i=0; i<100; i++)
    t[i] = p[i];
}

void b(void)
{
  char t[100];

  c(t);
}

int main(void)
{
  init_mark_malloc();

  aam();
  b();
  bbm(); /* this bbm is not absolutly necessary in this
	  * example */

  return 0;
}
And we would build it this way (the libmark_malloc.so is in the current directory) :
/home/roux/jmi/mark_malloc/release>gcc -g -o test test.c -L. -lmark_malloc
If you compile it with -Wall for example, you probably will get some warnings from the compiler, because of undeclared functions, but don't worry, all is fine. (One could declare the mark_malloc functions in the beggining of the program to avoid this minor problem.)

And now, we run (we must set the LD_LIBRARY_PATH environment variable for ld.so to find libmark_malloc.so at loading time) :

/home/roux/jmi/mark_malloc/release>export LD_LIBRARY_PATH=.:$LD_LIBRARY_PATH 
/home/roux/jmi/mark_malloc/release>./test 
The output result would be the same as previously.

Conclusion

Mark_malloc is a non-intrusive memory leaks detector. It can be used as an intrusive detector too, upon the choice of the user.

Its main feature is the backtrace capability, which gives a huge amount of information to the developper to check his errors. When I use gdb for instance, I always use the "bt" command, and the "up" one, to see where errors come from. Here, that's what you get.

It is limited in it does not give the parameters to the calling sequence, which I very often need when I debug with gdb. And when you've got the traces, it is a pain to go back to the input file and check the line by hand. A good extension would be to have a graphical interface which would let the coder, by a simple click of mouse, go to the file/line she wants.

Et voilà !
Don't hesitate to contact me : sed@free.fr for any reason.
Enjoy your life.
Last update : Thu Mar 14 16:05:08 CET 2002