Non-DLL function pointer call in DLL code

In DLL code, it is assumed that a function pointer points to a function descriptor. A function pointer call is made by first obtaining the function address through dereferencing the pointer; and then, branching to the function entry. When a non-DLL function pointer is passed to DLL code, it points directly to the function entry. An attempt to dereference through such a pointer produces an undefined function address. The subsequent branching to the undefined address may result in an exception.

The following is an example of passing a non-DLL function pointer to DLL code using an external variable. Its behavior is undefined as shown in Figure 1.

Figure 1. C non-DLL code

#include <stdio.h>
extern void (*fp)(void);
void hello(void) {
  printf("hello\n");
}
void goo(void) {
  fp = hello; /* assign address of hello, to fp    */
              /* (refer to
Figure 2). */
}

Figure 2 shows how dereferencing through a pointer produces an undefined function address in C.

Figure 2. C DLL code

extern void goo(void);
void (*fp)(void);
void main (void) {
  goo();
  (*fp)();    /* Expect a descriptor, but get a function address, */
              /* so it dereferences to an undefined address and   */
              /* call fails                                       */
}

Figure 3 shows how dereferencing through a pointer produces an undefined function address in C++.

Figure 3. C++ DLL code

extern "C" void goo(void);
void (*fp)(void);
void main (void) {
  goo();
  (*fp)();    /* Expect a descriptor, but get a function address, */
              /* so it dereferences to an undefined address and   */
              /* call fails                                       */
}

In Figure 4, a non-DLL function pointer call to an assembler function is resolved.

Figure 4. C++ DLL code calling an Assembler function

/*
 * This function must be compiled as DLL(CBA)
 */

extern "OS" {
  typedef void OS_FP(char *, int *);
}
extern "OS" OS_FP* ASMFN(char*);

int CXXFN(char* p1, int* p2) {
   OS_FP* fptr;

   fptr = ASMFN("ASM FN"); /* returns pointer to address of function */
   if (fptr) {
     fptr(p1, p2); /* call asm function through fn pointer */
   }
   return(0);
}