Memory Management

If you have ever received an error from overwriting storage created with the malloc() function, the following code may be of interest. It shows how to use debuggable versions of malloc()/calloc()/realloc() and free(). You can tailor the following macros.

Figure 1 shows an example program (CCNGMI1) that uses debuggable versions of malloc()/calloc()/realloc() and free() macros.

Figure 1. Debuggable malloc()/calloc()/realloc()/free() example
/* debuggable malloc()/calloc()/realloc()/free() example */
/* part 1 of 2-other file is CCNGMI2 */
#ifndef __STORAGE__
  #define __STORAGE__

  #define PADDING_SIZE         4    /* amount of padding around   */
                                    /* allocated storage          */
  #define PADDING_BYTE      0xFE    /* special value to initialize*/
                                    /* padding to                 */
  #define HEAP_INIT_SIZE    4096    /* get 4K to start with       */
  #define HEAP_INCR_SIZE    4096    /* get 4K increments          */
  #define HEAP_OPTS           72    /* HEAP(,,ANYWHERE,FREE)      */

  extern int heapVerbose;           /* If 0, heap allocation and  */
                                    /* free messages will be      */
                                    /* suppressed, otherwise, they*/
                                    /* will be displayed          */
#endif

Figure 2 shows the main routine (CCNGMI2) that calls the preceding macros.

Figure 2. Main routine - debuggable malloc()/calloc()/realloc()/free() example
/* debuggable malloc()/calloc()/realloc()/free() example */
/* part 2 of 2-other file is CCNGMI1 */
/*
 * STORAGE:
 *
 * EXTERNALS:
 *
 *  This file contains code for the following functions:
 *   -debug_malloc......allocate storage from a Language Environment heap
 *   -debug_calloc......allocate storage from a Language Environement heap
 *                and initialize it to 0.
 *   -debug_realloc.....re-allocate storage previously allocated
 *                by debug_malloc in this file. If a NULL pointer is passed
 *                instead of a previously allocated pointer, 
 *                debug_malloc will be called directly.
 *   -debug_free........free storage previously allocated by debug_malloc
 *                in this file.
 *  The prefix 'debug_' make sure these functions don't affect the calls
 *  to original functions within the libraries a user has no control over.
 *
 * USAGE:
 *
 *  To use this code, compile with no special options (although the
 *  DEBUG	option is useful so that the trace back will give
 *  additional information - line number information and the type and
 *  values of variables will be dumped in a trace back for all
 *  files compiled with DEBUG).
 *  Prelink (or link) this text deck with your text decks (make sure
 *  you explicitly link this text deck - avoid using autocall since
 *  you might get the C/370 version of malloc/free/realloc).
 *
 * INTERNALS:
 *
 *  General Algorithm:
 *
 *  When storage is allocated, extra 'padding' is allocated at the
 *  start and end of the actual storage allocated for the user.
 *  This padding is then initialized to a special pad value. If the
 *  user's code is functioning correctly, the padding should not
 *  have been changed when it comes time to free the storage. If the
 *  debug_free() routine finds that the padding does not have the correct
 *  value, the storage about to be freed is dumped and a trace back
 *  is issued, and then the storage is dumped, as usual.
 *  The padding size and padding byte value can be modified to suit
 *  your needs. Update the include file "ccngmi1.h" if you want
 *  to modify these values.
 *  Here is a diagram of how storage is allocated (assume that the
 *  pad value is xFE, the padding size is 4 bytes and 8 bytes of
 *  storage were requested):

 *  Length of      Padding        Allocated storage          Padding
 *   storage          |            returned to user             |
 *      |             |                     |                   |
 * +----+------+ +----+------+ +------------+------------+ +----+-----+
 * |           | |           | |                         | |          |
 *+--------------------------------------------------------------------+
 *| 00 00 00 10 | FE FE FE FE | xx xx xx xx | xx xx xx xx | FE FE FE FE|
 *+--------------------------------------------------------------------+
 *
 *  (Values above shown in hexadecimal)
 *
 *  This method is fairly effective in tracking down storage
 *  allocation problems. Also, code does not have
 *  to be recompiled to use these routines - it just has to be
 *  relinked. Note that it is not guaranteed to find all storage
 *  allocation errors - if you overwrite the padding with the
 *  same value it had before, or you overwrite more storage than
 *  you had padding for, you will still have problems.
 *
 *  This code uses the LE/370 heap services to allocate, re-allocate
 *  and free storage. A User Heap is used instead of the library
 *  heap so that if the heap gets corrupted, the standard library
 *  services that themselves use the heap won't be affected (i.e.
 *  if the user heap is damaged, a call to a library function
 *  such as printf should still succeed).
 *
 *  Notes of interest:
 *   - The runtime option STORAGE is very useful for tracking down
 *     random pointer problems - it initializes heap and/or stack frame
 *     storage to a particular value.
 *   - The runtime option RPTSTG(ON) is useful for improving heap and
 *     stack frame allocation - it generates a report indicating how
 *     stack and heap storage was managed for a given program.
 */
#include "ccngmi1.h"
#include <leawi.h>
#include <stdio.h>

/*
 * heapVerbose: external variable that controls whether heap
 *              allocation and free messages are displayed.
 */
int heapVerbose=1;

/*
 * mallocHeapID: static variable that is the Heap ID used for allocating
 *               storage via debug_malloc(). 
 *               On the first call to debug_malloc(), a Heap will be created 
 *               and this Heap ID will be set.
 *               All subsequent calls to debug_malloc will use this Heap ID.
 */
static _INT4 mallocHeapID=0;
/*
 * CHARS_PER_LINE/BYTES_PER_LINE: Used by dump() and DumpLine()
 *                                to control the width of a storage dump.
 */
#define CHARS_PER_LINE           40
#define BYTES_PER_LINE           16

/*
 * align: Given a value and the alignment desired (in bits), round
 *        the value to the next largest alignment, unless it is
 *        already aligned, in which case, just return the value passed.
 */
#pragma inline(align)

static int align(int value, int shift) {
  int alignment = (0x1 << shift);

  if (value % alignment) {
    return(((value >> shift) << shift) + alignment);
  }
  else {
    return(value);
  }
}
/*
 * padding: given a buffer (address and length), return 1 if the
 *          entire buffer consists of the pad character specified,
 *          otherwise return 0.
 */
#pragma inline(padding)
static int padding(const char* buffer, long size, int pad) {
  int i;
  for (i=0;i<size;++i) {
    if (buffer??(i??) != pad) return(0);
  }
  return(1);
}
/*
 * CEECmp: Given two feedback codes, return 0 if they have the same
 *         message number and facility id, otherwise return 1.
 */
#pragma inline(CEECmp)
static int CEECmp(_FEEDBACK* fc1, _FEEDBACK* fc2) {

  if (fc1->tok_msgno == fc2->tok_msgno &&
      !memcmp(fc1->tok_facid, fc2->tok_facid,
              sizeof(fc1->tok_facid))) {
    return(0);
  }
  else {
    return(1);
  }
}
/*
 * CEEOk: Given a feedback code, return 1 if it compares the same
to
 *        condition code CEE000.
 */
#pragma inline(CEEOk)
static int CEEOk(_FEEDBACK* fc) {
  _FEEDBACK CEE000 = { 0, 0, 0, 0, 0, {0,0,0}, 0 };

  return(CEECmp(fc, &CEE000) == 0); 
}

/*
 * CEEErr: Given a title string and a feedback code, print the
 *         title to stderr, then print the message associated
 *         with the feedback code. If the feedback code message can
not
 *         be printed out, print out the message number and severity.
 */
static void CEEErr(const char* title, _FEEDBACK* fc) {
  _FEEDBACK msgFC;
  _INT4 dest = 2;

  fprintf(stderr, "\n%s\n", title);
  CEEMSG(fc, &dest, &msgFC);

  if (!CEEOk(&msgFC)); {
    fprintf(stderr, "Message number:%d with severity %d occurred\n",
            fc->tok_msgno, fc->tok_sev);
  }
}
/*
 * DumpLine: Dump out a buffer (address and length) to stderr.
 */
static void DumpLine(char* address, int length) {
  int i, c, charCount=0;

  if (length % 4) length += 4;

  fprintf(stderr, "%8.8p: ", address);
  for (i=0; i < length/4; ++i) {
    fprintf(stderr, "%8.8X ", ((int*)address)??(i??));
    charCount += 9;
  }
  for (i=charCount; i < CHARS_PER_LINE; ++i) {
    putc(' ', stderr);
  }
  fprintf(stderr, "| ");
  for (i=0; i < length; ++i) {
    c = address??(i??);
    c = (isprint(c) ? c : '.');
    fprintf(stderr, "%c", c);
  }
  fprintf(stderr, "\n");
} 
/*
 * dump: dump out a buffer (address and length) to stderr by dumping out
 *       a line at a time (DumpLine), until the buffer is written out.
 */
static void dump(void* generalAddress, int length) {
  int curr = 0;
  char* address = (char*) generalAddress;

  while (&address??(curr??) <&address??(length-BYTES_PER_LINE??)) {
    DumpLine(&address??(curr??), BYTES_PER_LINE);
    curr += BYTES_PER_LINE;
  }
  if (curr < length) {
    DumpLine(&address??(curr??), length-curr);
  }
}
/*
 * debug_malloc: Create a heap if necessary by calling CEECRHP. This only
 *         needs to be done on the first call to debug_malloc(). Verify
 *         that the heap creation was ok. If it wasn't, issue an
 *         error message and return a NULL pointer.
 *         Write a message to stderr indicating how many bytes
 *         are about to be allocated.
 *         Call CEEGTST to allocate the storage requested plus
 *         additional padding to be placed at the start and end
 *         of the allocated storage. Verify that the storage allocation
 *         was successful. If it wasn't, issue an error message and
 *         return a NULL pointer.
 *         Write a message to stderr indicating the address of the
 *         allocated storage.
 *         Initialize the padding to the value of PADDING_BYTE, so that
 *         debug_free() will be able to test that the padding was not changed.
 *         Return the address of the allocated storage (starting after
 *         the padding bytes).
 */

void* debug_malloc(long initSize) {
  _FEEDBACK fc;
  _POINTER address=0;
  long totSize;
  long* lenPtr;
  char* msg;
  char* start;
  char* end;
 
  if (!mallocHeapID) {
    _INT4 heapSize = HEAP_INIT_SIZE;
    _INT4 heapInc  = HEAP_INCR_SIZE;
    _INT4 opts     = HEAP_OPTS;

    CEECRHP(&mallocHeapID, &heapSize, &heapInc, &opts, &fc);
    if (!CEEOk(&fc)) {
      CEEErr("Heap creation failed", &fc);
      return(0);
    }
  }
  if (heapVerbose) {
    fprintf(stderr, "Allocate %d bytes", initSize);
  }
  /*
   * Add the padding size to the total size, then round up to the
   * nearest double word
   */
  totSize  = initSize + (PADDING_SIZE*2) + sizeof(long);
  totSize  = align(totSize, 3);

  CEEGTST(&mallocHeapID, &totSize, &address, &fc);
  if (!CEEOk(&fc)) {
    msg = "Storage request failed";
    CEEErr(msg, &fc);
    __ctrace(msg);

    return(0);
  }
  lenPtr = (long*) address;
  *lenPtr= initSize;
  start  = ((char*) address) + sizeof(long);
  end    = start + initSize + PADDING_SIZE;
  memset(start, PADDING_BYTE, PADDING_SIZE);
  memset(end,   PADDING_BYTE, PADDING_SIZE);

  if (heapVerbose) {
    fprintf(stderr, " starting at address %p\n", address);
  }

  return(start + PADDING_SIZE);
}
/*
 * debug_calloc: Call debug_malloc() to allocate the requested amount
 *         of storage. If the allocation was successful, 
 *         initialize the allocated storage to 0.
 *         Return the address of the allocated storage (or a NULL
 *         pointer if debug_malloc returned a NULL pointer).
 */
  void* debug_calloc(size_t num, size_t size) {
  size_t initSize = num * size;
  void* ptr;
  ptr = debug_malloc(initSize);
  if (ptr) {
    memset(ptr, 0, initSize);
  }
  return(ptr);
}
/*
 * debug_realloc: If a NULL pointer is passed, call debug_malloc() directly.
 *          Call CEECZST to re-allocate the storage requested plus
 *          additional padding to be placed at the start and end
 *          of the allocated storage.
 *          Verify that the storage re-allocation was ok. If it wasn't,
 *          issue an error message, dump the storage, and return a NULL
 *          pointer.
 *          Write a message to stderr indicating the address of the
 *          re-allocated storage.
 *          Initialize the padding to the value of PADDING_BYTE, so
 *          that debug_free() will be able to test that the padding was not
 *          changed. Note that the padding at the start of the storage
 *          does not need to be allocated, since it was already
 *          initialized by an earlier call to debug_malloc().
 *          Return the address of the re-allocated storage (starting
 *          after the padding bytes).
 */

void* debug_realloc(char* ptr, long initSize) {
  _FEEDBACK fc;
  _POINTER address = (ptr - sizeof(long) - PADDING_SIZE);

  long oldSize;
  long* lenPtr;
  char* start;
  char* end;
  char* msg;
  long newSize = initSize;

  if (ptr == 0) {
    return(debug_malloc(newSize));
  }

  oldSize = *((long*) address);

  if (heapVerbose) {
    fprintf(stderr, "Re-allocate %d bytes from address %p to ",
            newSize, address);
  }

  /*
   * Add the padding size to the total size, then round up to the
   * nearest double word
   */

  newSize += (PADDING_SIZE*2) + sizeof(long);
  newSize  = align(newSize, 3);
  CEECZST(&address, &newSize, &fc);
  if (!CEEOk(&fc)) {
    msg = "Storage re-allocation failed";

    CEEErr(msg, &fc);
    dump(address, oldSize + (PADDING_SIZE*2) + sizeof(long));
    __ctrace(msg);
    return(0);
  }

  lenPtr = (long*) address;
  *lenPtr= initSize;
  start  = ((char*) address) + sizeof(long);
  end    = start + initSize + PADDING_SIZE;

  memset(end, PADDING_BYTE, PADDING_SIZE);
  if (heapVerbose) {
    fprintf(stderr, "address %p\n", address);
  }
  return(start + PADDING_SIZE);
}
/*
 * debug_free: Calculate where the start and end of the originally
 *       allocated storage was. The start will be different than the
 *       address passed in because the address passed in points after
 *       the padding bytes added by debug_malloc() or debug_realloc().
 *       Write a message to stderr indicating what address is about
 *       to be freed.
 *       Verify that the start and end padding bytes have the original
 *       padding value. If they don't, dump out the originally
 *       allocated storage and issue a trace.
 *       Free the storage by calling CEEFRST. If the storage free
 *       fails, dump out the storage and issue a trace.
 */
void debug_free(char* ptr) {
  _FEEDBACK fc;
  _POINTER address=(void*) (ptr - sizeof(long) - PADDING_SIZE);
  char* start;
  char* end;
  long size;
  long* lenPtr;
  char* msg;
  lenPtr = (long*) address;
  size   = *lenPtr;
  start  = ((char*) address) + sizeof(long);
  end    = start + size + PADDING_SIZE;

  if (heapVerbose) {
    fprintf(stderr, "Free address %p\n", address);
  }
  if (!padding(start, PADDING_SIZE, PADDING_BYTE) ||
      !padding(end, PADDING_SIZE, PADDING_BYTE)) {

    dump(address, size + (PADDING_SIZE*2) + sizeof(long));
    msg = "Padding overwritten";
    __ctrace(msg);
  }
  else {
    CEEFRST(&address, &fc);
    if (!CEEOk(&fc)) {
      msg = "Storage debug_free failed";

      CEEErr(msg, &fc);
      dump(address, size + (PADDING_SIZE*2) + sizeof(long));
      __ctrace(msg);
    }
  }
}