.NETInteroperability Between .NET & Win32

Brian Long (www.blong.com)

Table of Contents

Click here to download the files associated with this article.

If you find this article useful then please consider making a donation. It will be appreciated however big or small it might be and will encourage Brian to continue researching and writing about interesting subjects in the future.


Introduction

.NET is a new programming platform representing the future of Windows programming. Developers are moving across to it and learning the new .NET oriented languages and frameworks, but new systems do not appear overnight. It is a lengthy process moving entire applications across to a new platform and Microsoft is very much aware of this.

To this end .NET supports a number of interoperability mechanisms that allow applications to be moved across from the Win32 platform to .NET piece by piece, allowing developers to still build complete applications, but which comprise of Win32 portions and some .NET portions, of varying amounts.

When building new .NET applications, there are provisions for using existing Win32 DLL exports (both custom DLL routines and standard Win32 API routines) as well as COM objects (which then act like any other .NET object).

When building Win32 applications there is a process that allows you to access individual routines in .NET assemblies. When building Win32 COM client applications, there is a mechanism that lets you use .NET objects as if they were normal COM objects.

This paper investigates the interoperability options that do not involve COM:

  • .NET assemblies accessing Win32 DLL exports (including Win32 APIs)
  • Win32 applications/DLLs accessing routines in .NET assemblies

The accompanying paper, .NET Interoperability: COM Interop (see Reference 1), looks at interoperability between .NET and COM (termed COM Interop).

The coverage will have a specific bias towards a developer moving code from Borland Delphi (for Win32) to Borland Delphi 8 for .NET, however the principles apply to any other development tools. Clearly the Delphi-specific details will not apply to other languages but the high-level information will still be relevant.

Because of the different data types available on the two platforms (such as PChar in Win32 and the new Unicode String type on .NET), inter-platform calls will inevitably require some form of marshaling process to transform parameters and return values between the data types at either end of the call. Fortunately, as we shall see, the marshalling is done for us after an initial process to set up the inter-platform calls.

.NET Clients Using Win32 DLL Exports (P/Invoke)

This is the most common form of interoperability, which is why we are looking at it first. Whilst the .NET Framework is large and wide ranging, there are still things that you can do using Win32 APIs that are not possible using just the .NET framework. Simple examples include producing noises (using the Win32 APIs MessageBeep and Beep) or performing high accuracy timing (with QueryPerformanceCounter and QueryPerformanceFrequency). Note that the first example cited here, making noises, will be rectified in .NET 2.0 as support is to be added for this.

In cases like this, where it would be helpful, or indeed necessary, to make use of a Win32 DLL routine from a .NET application you use a mechanism called the Platform Invocation Service, which is usually referred to as Platform Invoke or simply P/Invoke (or PInvoke). This service operates through a custom attribute, DllImportAttribute, defined in the System.Runtime.InteropServices namespace. The attribute allows the name of the implementing DLL (and other necessary information) to be associated with a procedure or function declaration, thereby allowing the DLL routine to be called.

The marshaling of parameters and return values between the managed .NET world and the unmanaged Win32 world is automatically performed by the Interop Marshaler used by the COM Interop support.

For an example, consider a Win32 DLL that exports three routines with the following signatures:

 
function DoSomething(I: Integer): Bool; cdecl;
function DoSomethingElseA(Msg: PChar): Bool; cdecl;
function DoSomethingElseW(Msg: PWideChar): Bool; cdecl;

As you can see, they all use the C calling convention, rather than the usual Win32 standard calling convention. The first routine takes an integer and returns a Boolean value, though using the standard Windows type Bool (a 32-bit Boolean value, equivalent to LongBool). The second routine takes a pointer to an ANSI character string (PChar is the same as PAnsiChar) and the last takes a pointer to a Unicode string.

To construct an appropriate import declaration that uses the P/Invoke mechanism you have two options, using the traditional Delphi DLL import syntax or using the custom P/Invoke attribute.

Traditional Syntax

You can use historic Delphi import declaration syntax and completely ignore the custom attribute, although there are caveats to this. We must understand the implications of leaving out the custom attribute in doing this. In point of fact, Delphi for .NET will create a custom attribute behind the scenes and the key thing is to understand what values the attribute fields will take.

The first exported routine can be declared in a .NET import unit like this:

 
unit Win32DLLImport;
 
interface
 
function DoSomething(I: Integer): Boolean; cdecl;
 
implementation
 
const
  Win32DLL = 'Win32DLL.dll';
 
function DoSomething(I: Integer): Boolean;
external Win32DLL name 'DoSomething';
 
end.

The calling convention is specified using the standard cdecl directive in the declaration part, and the DLL name and optional real DLL export name are specified in the implementation part.

This works fine for routines that do not have textual parameter types or return types. Internally, the compiler massages the declaration to use the P/Invoke attribute, like this:

 
function DoSomething(I: Integer): Boolean;
...
[DllImport(Win32DLL, CallingConvention = CallingConvention.Cdecl)]
function DoSomething(I: Integer): Boolean;
external;

Note that as well as the attribute constructor parameter (the DLL name) the attribute also has a CallingConvention field (attribute fields are often called parameters) that is set to specify the C calling convention. Indeed there are other parameters available in the attribute, which assume default values, and that is where problems can arise when the routine uses textual parameters or return types.

Attribute Syntax

The alternative to using traditional syntax is to explicitly specify the P/Invoke attribute in the import declaration. This will often be necessary when the routine takes textual parameters due to the default value of the attribute's CharSet parameter.

CharSet can take these values:

  • Ansi: textual parameters are treated as ANSI strings.
    This is the default value if you declare a DllImport attribute, but omit the
    CharSet parameter.
  • Auto: textual parameters are treated as ANSI strings on Win9x platforms and Unicode strings on NT-based systems (WinNT/2K/XP). This is the default setting generated by the compiler when it silently manufactures the whole DllImport attribute on your behalf.
  • None: this is considered obsolete but has the same effect as Ansi
  • Unicode: textual parameters are treated as Unicode strings

It is common for custom DLL routines to be implemented to take a fixed string type (either ANSI or Unicode), typically ANSI as Unicode is not implemented on Win9x systems. The same DLL will be deployed on any Windows system.

On the other hand, Win32 APIs that take string parameters are implemented twice; one implementation has an A suffix (the ANSI one) and one has a W suffix (the Unicode one). On Win9x systems the Unicode implementation is stubbed out as Unicode is not supported.

If the CharSet field is set to Ansi and the routine you are declaring is called Foo, at runtime the P/Invoke system will look for Foo in the DLL and use it if found; if it is not found it will look for FooA. However if CharSet is set to Unicode then FooW will be sought first, and if it is not found Foo will be used, if present.

The Auto value for CharSet means that on Win9x systems, string parameters will be turned into ANSI strings and the ANSI entry point search semantics will be used. On NT-based systems the parameters will be turned to Unicode and the Unicode search semantics will be used. For normal Win32 APIs this is just fine, but for most custom DLL routines a specific CharSet value must be specified.

In the case of the sample DLL exports shown above, we have two implementations of a routine, one that takes an ANSI parameter and one that takes a Unicode parameter. These are named following the Win32 conventions and so we could define a single .NET import like this, which would work fine on all Windows systems, calling DoSomethingElseA or DoSomethingElseW based on the Windows system type:

 
function DoSomethingElse(const Msg: String): Boolean; cdecl;
...
function DoSomethingElse(const Msg: String): Boolean;
external Win32DLL;

This is the same as explicitly writing the P/Invoke attribute like this:

 
function DoSomethingElse(const Msg: String): Boolean;
...
[DllImport(Win32DLL, EntryPoint = 'DoSomethingElse', CharSet = CharSet.Auto, CallingConvention = CallingConvention.Cdecl)]
function DoSomethingElse(const Msg: String): Boolean;
external;

If we didn't have both an ANSI and Unicode implementation, and instead had just an ANSI version, then the single import declaration would look something like this:

 
function DoSomethingElse(const Msg: String): Boolean;
...
[DllImport(Win32DLL, EntryPoint = 'DoSomethingElse', CharSet = CharSet.Ansi, CallingConvention = CallingConvention.Cdecl)]
function DoSomethingElse(const Msg: String): Boolean;
external;

We can now define two specific .NET imports for the ANSI and Unicode versions of the routine as follows:

 
function DoSomethingElseA(const Msg: String): Boolean;
function DoSomethingElseW(const Msg: WideString): Boolean;
...
[DllImport(Win32DLL, CharSet = CharSet.Ansi, CallingConvention = CallingConvention.Cdecl)]
function DoSomethingElseA(const Msg: String): Boolean;
external;
 
[DllImport(Win32DLL, CharSet = CharSet.Unicode, CallingConvention = CallingConvention.Cdecl)]
function DoSomethingElseW(const Msg: String): Boolean;
external;

Note that the default value of the CallingConvention parameter is StdCall when you use DllImportAttribute and omit it. However when you use traditional Delphi syntax and do not specify a calling convention, Delphi specifies a calling convention of WinApi, which is equivalent to StdCall on Windows, but equivalent to Cdecl on Windows CE (which is not supported as a target CLR version by Delphi 8 for .NET).

Working Out The Parameter Types

In this case the .NET equivalent of the original data types was quite straightforward: PChar, PAnsiChar and PWideChar become String, Bool becomes Boolean and Integer becomes Integer. In other cases, the corresponding types to use in the .NET import declaration may not be so clear, particularly if the original declaration was written in C.

In many cases the appropriate information can be obtained by finding a normal Win32 API that uses the same parameter type in the same way and looking up the declaration in the Win32 import unit supplied with Delphi for .NET, Borland.Vcl.Windows.pas (in Delphi for .NET's Source\rtl directory). This unit contains import declarations for the majority of the standard Win32 APIs.

For example, consider an existing API or two that we can test the theory out with: GetComputerName and GetUserName. If these were part of some third party DLL, targeted at C/C++ programmers, which we were using in our Win32 applications, then we may well want to use them in a .NET application. The C declarations of these routines look like:

 
BOOL GetComputerName(
    LPTSTR lpBuffer,   // address of name buffer 
    LPDWORD nSize      // address of size of name buffer 
   );
BOOL GetUserName(
    LPTSTR lpBuffer,   // address of name buffer 
    LPDWORD nSize      // address of size of name buffer 
   );

Since you have already used them in your Win32 applications you will already have Delphi translations of these (which in this example's case we can get from the Delphi 7 Windows.pas import unit):

 
function GetComputerName(lpBuffer: PChar; var nSize: DWORD): BOOL; stdcall;
function GetUserName(lpBuffer: PChar; var nSize: DWORD): BOOL; stdcall;

In both cases the lpBuffer parameter is an out parameter that points to a buffer (a zero-based Char array) that receives the string with the computer's or user's name. The nSize parameter is an in/out parameter that specifies how large the buffer is, so the routine doesn't write past the end of it. If the buffer is too small, the routine returns False, otherwise it returns how many characters were written to the buffer.

If the documentation for the routine tells you the maximum size of the returned string you can easily make the buffer that large, otherwise you will have to check the return value; if it fails try a larger buffer.

There are many Win32 routines that take parameters that work this or a similar way, such as GetWindowsDirectory, GetSystemDirectory and GetCurrentDirectory. Sometimes the routine returns the number of characters (either that it wrote, or that it requires) and the buffer size parameter is passed by value (as in the routines just referred to), other times the function returns a Boolean value and the buffer size parameter is passed by reference. Win32 import declarations for these last three routines look like this:

 
function GetWindowsDirectory(lpBuffer: PChar; uSize: UINT): UINT; stdcall;
function GetSystemDirectory(lpBuffer: PChar; uSize: UINT): UINT; stdcall;
function GetCurrentDirectory(nBufferLength: DWORD; lpBuffer: PChar): DWORD; stdcall;

The corresponding Delphi for .NET declarations for all these routines can be found in Borland.Vcl.Windows.pas. The declarations in the interface section look like this:

 
function GetUserName(lpBuffer: StringBuilder; var nSize: DWORD): BOOL;
function GetComputerName(lpBuffer: StringBuilder; var nSize: DWORD): BOOL;
function GetWindowsDirectory(lpBuffer: StringBuilder; uSize: UINT): UINT;
function GetSystemDirectory(lpBuffer: StringBuilder; uSize: UINT): UINT;
function GetCurrentDirectory(nBufferLength: DWORD; lpBuffer: StringBuilder): DWORD;

As you can see, these types of string parameters are best represented using StringBuilder objects. StringBuilder is an appropriate type when the underlying Win32 routine will modify the string buffer, whereas String can be used when the routine will not modify its content (.NET String objects are immutable).

StringBuilder objects must have their capacity set to your desired size and that capacity can then be passed as the buffer size. The following five event handlers show how each of these APIs can be called from Delphi for .NET through P/Invoke.

 
procedure TfrmPInvoke.btnUserNameClick(Sender: TObject; Args: EventArgs);
var
  UserBuf: StringBuilder;
  UserBufLen: DWord;
begin
  UserBuf := StringBuilder.Create(64);
  UserBufLen := UserBuf.Capacity;
  if GetUserName(UserBuf, UserBufLen) then
    MessageBox.Show(UserBuf.ToString)
  else
    //User name is longer than 64 characters
end;
 
procedure TfrmPInvoke.btnComputerNameClick(Sender: TObject; Args: EventArgs);
var
  ComputerBuf: StringBuilder;
  ComputerBufLen: DWord;
begin
  //Set max size buffer to ensure success
  ComputerBuf := StringBuilder.Create(MAX_COMPUTERNAME_LENGTH);
  ComputerBufLen := ComputerBuf.Capacity;
  if GetComputerName(ComputerBuf, ComputerBufLen) then
    MessageBox.Show(ComputerBuf.ToString)
end;
 
procedure TfrmPInvoke.btnWindowsDirClick(Sender: TObject; Args: EventArgs);
var
  WinDirBuf: StringBuilder;
begin
  WinDirBuf := StringBuilder.Create(MAX_PATH); //Set max size buffer to ensure success
  GetWindowsDirectory(WinDirBuf, WinDirBuf.Capacity);
  MessageBox.Show(WinDirBuf.ToString)
end;
 
procedure TfrmPInvoke.btnSystemDirClick(Sender: TObject; Args: EventArgs);
var
  SysDirBuf: StringBuilder;
begin
  SysDirBuf := StringBuilder.Create(MAX_PATH); //Set max size buffer to ensure success
  GetSystemDirectory(SysDirBuf, SysDirBuf.Capacity);
  MessageBox.Show(SysDirBuf.ToString)
end;
 
procedure TfrmPInvoke.btnCurrentDirClick(Sender: TObject; Args: EventArgs);
var
  CurrDirBuf: StringBuilder;
begin
  CurrDirBuf := StringBuilder.Create(MAX_PATH); //Set max size buffer to ensure success
  GetCurrentDirectory(CurrDirBuf.Capacity, CurrDirBuf);
  MessageBox.Show(CurrDirBuf.ToString)
end;

In addition to the Delphi Win32 import unit you can also find C# P/Invoke declarations for much of the common Win32 API in Appendix E of .NET and COM (see Reference 2) and also in a dedicated online repository (see Reference 3).

Win32 Errors

Win32 routines often return False or 0 to indicate they failed (the documentation clarifies whether this is the case), leaving the programmer to call GetLastError to find the numeric error code. Delphi programmers can call SysErrorMessage to turn the error number into an error message string to do with as they will or call RaiseLastWin32Error or RaiseLastOSError to raise an exception with the message set to the error message for the last error code. Additionally Delphi offers the Win32Check routine that can take a Win32 API Boolean return value; this calls RaiseLastOSError if the parameter is False.

It is important that when calling Win32 routines from .NET you do not declare or use a P/Invoke declaration for the Win32 GetLastError API as it is unreliable (due to the interaction between .NET and the underlying OS). Instead you should use Marshal.GetLastWin32Error from the System.Runtime.InteropServices namespace. This routine relies on another DllImportAttribute field being specified. The SetLastError field defaults to False meaning the error code is ignored. If set to True the runtime marshaler will call GetLastError and cache the value for GetLastWin32Error to return.

Note that all the Win32 imports in Delphi for .NET specify this field with the value of True. For example, this is the GetComputerName declaration from the Borland.Vcl.Windows.pas implementation section:

 
const
  kernel32  = 'kernel32.dll';
 
[DllImport(kernel32, CharSet = CharSet.Auto, SetLastError = True, EntryPoint = 'GetComputerName')]
function GetComputerName; external;

Also note that Delphi for .NET defines a routine GetLastError in the implicitly used Borland.Delphi.System unit, implemented simply as:

 
function GetLastError: Integer;
begin
  Result := System.Runtime.InteropServices.Marshal.GetLastWin32Error;
end;

However, if you use Borland.Vcl.Windows and call GetLastError in the same unit, the compiler will bind the call to the version of GetLastError defined in that unit. However thankfully it is not a P/Invoke declaration to the Win32 routine of that name. Instead it is a simple function that calls Borland.Delphi.System.GetLastError.

 
function GetLastError: DWORD;
begin
  Result := Borland.Delphi.System.GetLastError;
end;

To aid moving API-based code across, Borland.Vcl.SysUtils contains ported versions of the Delphi Win32 error support routines, such as SysErrorMessage, RaiseLastWin32Error, RaiseLastOSError and Win32Check. So just as in regular Win32 Delphi applications you can write code like this:

 
procedure TfrmPInvoke.btnUserNameClick(Sender: TObject; Args: EventArgs);
var
  UserBuf: StringBuilder;
  UserBufLen: LongWord;
begin
  //Buffer too small, so we will get an exception
  UserBuf := StringBuilder.Create(2);
  UserBufLen := UserBuf.Capacity;
  Win32Check(GetUserName(UserBuf, UserBufLen));
  MessageBox.Show(UserBuf.ToString)
end;
 
procedure TfrmPInvoke.btnWindowsDirClick(Sender: TObject; Args: EventArgs);
var
  WinDirBuf: StringBuilder;
begin
  WinDirBuf := StringBuilder.Create(MAX_PATH); //Set max size buffer to ensure success
  Win32Check(Bool(GetWindowsDirectory(WinDirBuf, WinDirBuf.Capacity)));
  MessageBox.Show(WinDirBuf.ToString)
end;

If you are running this code in a WinForms application then the exceptions generated to represent the error will be picked up by the default WinForms unhandled exception handler. This takes the form of a dialog showing you details of the exception and offering you the chance to terminate or continue the application. You can safely continue from these exceptions.

If the Details button on this dialog is pressed you get a useful stack trace pointing you to the execution path that led to the exception.

HRESULT Return Values

As you may be aware, various COM/OLE related Win32 APIs return HResult values. These values return various bits of status information such as success, failure and also error codes. These APIs can be declared using the P/Invoke mechanism as well as any other method (HResults are represented as integers in .NET). For example, let's take the CLSIDFromProgID API, which is declared in Win32 terms as:

 
function CLSIDFromProgID(pszProgID: POleStr; out clsid: TCLSID): HResult; stdcall;
external ole32 name 'CLSIDFromProgID'

The first parameter is a POleStr (or PWideChar), meaning it is a Unicode string on all Windows platforms. In C terms the parameter type is LPCOLESTR, where the C implies the routine considers the string constant and will not change it (we can use a String parameter instead of a StringBuilder in the P/Invoke definition thanks to this fact).

One way of writing the P/Invoke import is using the standard Delphi syntax:

 
function CLSIDFromProgID([MarshalAs(UnmanagedType.LPWStr)] ppsz: String; out rclsid: Guid): Integer; stdcall;
external ole32;

Notice in this case that the String parameter needs an attribute of its own to ensure it will always be marshaled correctly on all Windows platforms. By default, a String parameter in this type of declaration will be marshaled as Unicode on NT platforms and ANSI on Win9x platforms. An alternative would be to specify the API uses the Unicode character set:

 
[DllImport(ole32, CharSet = CharSet.Unicode)]
function CLSIDFromProgID(ppsz: String; out rclsid: Guid): Integer;
external;

Delphi programmers may be familiar with the safecall calling convention that allows HResults to be ignored by the developer. Instead, safecall methods automatically raise exceptions if the HResult returned indicates failure.

P/Invoke supports a similar mechanism with yet another DllImportAttribute field, PreserveSig. This field defaults to True, meaning that the API signature will be preserved, thereby returning a HResult. If you set PreserveSig to False you can remove the HResult return value and a failure HResult will automatically raise an exception. The above declarations could be rewritten as:

 
[DllImport(ole32, PreserveSig = False)]
procedure CLSIDFromProgID([MarshalAs(UnmanagedType.LPWStr)] ppsz: String; out rclsid: Guid);
external;

or:

 
[DllImport(ole32, CharSet = CharSet.Unicode, PreserveSig = False)]
procedure CLSIDFromProgID(ppsz: String; out rclsid: Guid);
external;

Performance Issues

Sometimes there may be various ways to express a DLL routine in .NET and indeed the marshaling system will do its best to cope with the way you express the routine signature. However some representations (data mappings between types) are more efficient than others. Take the high accuracy timing routines for example. These are declared in Delphi 7 like this, where TLargeInteger is a variant record containing an Int64 (whose high and low parts can be accessed through other fields):

 
const
  kernel32  = 'kernel32.dll';
 
function QueryPerformanceCounter(var lpPerformanceCount: TLargeInteger): BOOL; stdcall;
external kernel32 name 'QueryPerformanceCounter';
function QueryPerformanceFrequency(var lpFrequency: TLargeInteger): BOOL; stdcall;
external kernel32 name 'QueryPerformanceFrequency';

The logical way of translating these routines would be like this:

 
function QueryPerformanceCounter(var lpPerformanceCount: Int64): Boolean;
external kernel32;
function QueryPerformanceFrequency(var lpFrequency: Int64): Boolean;
external kernel32;

This requires the marshaler to translate from a BOOL (which is the same a LongBool, a 32-bit Boolean value where all bits are significant) to a Boolean object. It would be more efficient to choose a data type that was the same size and can have the value passed straight through. Also, since the documentation for these APIs specifies that they will write a value to the reference parameter, and are not interested in any value passed in, we can replace the var declaration with out to imply this fact. So a more accurate and more efficient pair of declarations would look like this:

 
function QueryPerformanceCounter(out lpPerformanceCount: Int64): LongBool;
external kernel32;
function QueryPerformanceFrequency(out lpFrequency: Int64): LongBool;
external kernel32;

In a case like this where we are calling high performance timers you can go one step further to remove overheads by using the SuppressUnmanagedCodeSecurityAttribute from the System.Security namespace:

 
[DllImport(kernel32), SuppressUnmanagedCodeSecurity]
function QueryPerformanceCounter(out lpPerformanceCount: Int64): LongBool;
external;
[DllImport(kernel32), SuppressUnmanagedCodeSecurity]
function QueryPerformanceFrequency(out lpFrequency: Int64): LongBool;
external;

This makes the calls to the routines a little more efficient at the expense of normal security checks and thereby means the reported times from the routines will be slightly more accurate. A simple test shows that the logical versus accurate declaration have little to distinguish them at runtime, but the declaration with security disabled is a little quicker.

SuppressUnmanagedCodeSecurityAttribute should only be used on routines that cannot be used maliciously because, as the name suggests, it bypasses the normal runtime security check for calling unmanaged code. A routine marked with this attribute can be called by .NET code that does not have permission to run unmanaged code (such as code running via a Web browser page).

One additional performance benefit you can achieve, according to the available information on the subject, is to cause the P/Invoke signature/metadata validation, DLL loading and routine location to execute in advance of any of the P/Invoke routine calls. By default the first time a P/Invoke routine from a given DLL is called, that is when the DLL is loaded. Similarly, the signature metadata is validated the first time the P/Invoke routine is called. You can do this in advance by calling the Marshal.Prelink method (for a single P/Invoke routine) or the Marshal.PrelinkAll (for all P/Invoke routines defined in a class or unit). Both these come from the System.Runtime.InteropServices namespace.

The two timing routines are declared as standalone routines in the unit, but to fit into the .NET model of having everything defined as a method, this really means they are part of the projectname.Unit namespace (dotNetApp.Unit in this case). So to pre-link the two timing routines from a form constructor you could use:

 
Marshal.Prelink(GetType.Module.GetType('dotNetApp.Unit').GetMethod('QueryPerformanceFrequency'));
Marshal.Prelink(GetType.Module.GetType('dotNetApp.Unit').GetMethod('QueryPerformanceCounter'));

To pre-link all P/Invoke routines in the unit from the form constructor, use:

 
Marshal.PrelinkAll(GetType.Module.GetType('dotNetApp.Unit'));

Simple tests indicate that without the pre-link code, the first tests will indeed be a little slower than subsequent tests in the same program run. Pre-linking the specified routines individually removes this first-hit delay, but curiously I found calling PrelinkAll makes each test in a given session noticeably quicker than with any of the previous tests.

Win32 Clients Using Unmanaged Exports (Inverse P/Invoke)

The CCW aspect of COM Interop (see Reference 1) permits a COM client to access a .NET object just like any other COM object, but sometimes a Win32 application can benefit from being given access to just a handful of routines in a .NET assembly. This technique is catered for by the .NET platform; the CLR has an inherent capability to expose any .NET method to the outside (Win32) world using a mechanism that is the exact reverse of that used by P/Invoke, hence the term Inverse P/Invoke. However only Delphi for .NET, C++ With Managed Extensions (aka Managed C++) and the underlying Intermediate Language (IL) support this ability directly. There is also little printed coverage of this technique, other than in Chapter 15 of Inside Microsoft .NET IL Assembler (see Reference 5) and Chapter 16 of Delphi for .NET Developerís Guide (see Reference 4).

Exporting Global Routines

Delphi for .NET supports the Inverse P/Invoke mechanism using the same syntax as Delphi for Win32 supports exporting routines from DLLs, via the exports clause. This ability is unique in the more popular .NET languages (specifically C# and Visual Basic.NET do not support it) and is referred to as unmanaged exports. The syntax is limited to exporting Delphi global routines (implemented as class static methods of the unit wrapper class Unit that Delphi generates behind the scenes). Regular class static methods cannot be exported.

A sample assembly project source file is shown below. Notice that it is required to mark the project source as unsafe for the exports clause to compile. You can do this either with the compiler directive shown, or if compiling with the command-line compiler you can pass the --unsafe+ switch. Listing a routine for export is considered making the routine unsafe.

 

library dotNetAssembly;

 

{$UNSAFECODE ON}

 

{%DelphiDotNetAssemblyCompiler ' $(SystemRoot)\microsoft.net\framework\v1.1.4322\System.Windows.Forms.dll'}

 

uses

  System.Windows.Forms;

 

procedure DoSomething(I: Integer);

begin

  MessageBox.Show(I.ToString)

end;

 

procedure DoSomethingElse(const Msg: String);

begin

  MessageBox.Show(Msg)

end;

 

exports

  DoSomething,

  DoSomethingElse;

 

begin

end.
 

Limitations

As mentioned, the Delphi for .NET support for unmanaged exports is (unnecessarily) restricted to exporting global routines. You are not permitted to export an arbitrary class static method, even though global routines are implemented as class static methods as mentioned above. To explain, whilst we can write global routines as we also have been able to, Delphi for .NET will ensure they are implemented as class static methods, since all CLS-compliant routines in .NET must be methods. Every source file in a Delphi for .NET project has an implicit class, called Unit, containing everything in the source file. So the routines DoSomething and DoSomethingElse in the previous listing end up acting as though they were declared something like this:

 
type
  Unit = class(&Object)
  public
    class procedure DoSomething(I: Integer); static;
    class procedure DoSomethingElse(const Msg: String); static;
  end;

As an example of regular class static methods not being exportable the library project file listed below contains a sample class that declares a couple of class methods. There is no way to export these methods to unmanaged code using the unmanaged exports feature of Delphi 8 for .NET.

 

library dotNetAssembly;

 

{%DelphiDotNetAssemblyCompiler ' $(SystemRoot)\microsoft.net\framework\v1.1.4322\System.Windows.Forms.dll'}

 

uses

  System.Windows.Forms;

 

type

  SampleClass = class

  public

    class procedure DoSomething(I: Integer); static;

    class procedure DoSomethingElse(const Msg: String); static;

  end;

 

class procedure DoSomething(I: Integer);

begin

  MessageBox.Show(I.ToString)

end;

 

class procedure DoSomethingElse(const Msg: String);

begin

  MessageBox.Show(Msg)

end;

 

//This part wonít compile

//exports

//  SampleClass.DoSomething,

//  SampleClass.DoSomethingElse;

 

begin

end.

This shows up a limitation of Delphi for .NET unmanaged exports, but of course in C# and Visual Basic.NET the limitation is harsher. You cannot export static methods for use from unmanaged code at all. Fortunately the next section shows us how we can overcome these limitations if we wish to.

Removing Limitations With Creative Round Tripping

High-level .NET programming languages do not take full advantage of all the features of the CLR. The CLR feature set, accessible through IL is very rich, attempting to cater for as many requirements of different and disparate languages as possible. High level languages use whatever subset is needed to implement the language features and ignore the rest as immaterial. Sometimes developers might want to employ a feature or two of IL that is not accessible through their language of choice. In this case we would be looking for the mechanism to export arbitrary class static methods. We can achieve this by resorting to some cunning trickery, namely creative round tripping.

Round tripping is a term describing a two step process that involves taking a managed Win32 file (a .NET assembly) and disassembling it to the corresponding IL source code and metadata (and any managed or unmanaged resources), and then reassembling the IL code, metadata and resources into an equivalent .NET binary.

Because of the rich, descriptive nature of the metadata in managed PE (Portable Executable, the Win32 file format) files this round-tripping process is very reliable. A few select things do not survive the process, but these are not things that are supported by the Delphi for .NET compiler, for example, data on data (data that contains the address of another data constant) and embedded native, non-managed code. Also, local variable names will be lost if there is no PDB file available since they are only defined in debug information and not in metadata.

Round tripping in itself is only useful to prove that you get a working executable back after a disassembly/reassembly process. The term creative round tripping is used to describe a round tripping job with an extra step. After disassembling the assembly into IL code, you modify the IL code before reassembly.

Creative round tripping is used in these scenarios:

  • changing the IL code generated by a compiler in a way not permitted by the compiler, such as to export .NET class static methods
  • adding custom IL code to your classes
  • merging several modules into one module

Clearly we will be focusing on the first item in the list in order to overcome Delphi for .NETís unmanaged exports limitation, which will apply equally to the general unmanaged export limitation in C# or Visual Basic.NET. Let's first look at the two steps involved in round tripping to get the gist of things before looking at the details of creative round tripping.

Round Tripping, Step 1: Disassembly

To disassemble a .NET assembly you use the .NET Framework IL Disassembler, ildasm.exe, which comes with the Framework SDK. You will need to ensure the PATH has been set appropriately to allow you to run it without entering a fully-qualified path for the utility each time. You can do this globally using the Environment Variables button in the System Properties dialog or by running the handy batch file, SDKVars.bat, supplied with the .NET Framework SDK. You can find this batch file in the C:\Program Files\Microsoft.NET\SDK\v1.1\bin directory, assuming a default installation.

Note that the batch file will only execute the command prompt session that it is executed within and will not have a global effect and so it can be handy to set up a shortcut on the desktop or in the Start menu somewhere that executes this command line to invoke a command-prompt session and execute the batch file within it:

cmd /K "C:\Program Files\Microsoft.NET\SDK\v1.1\bin\SDKVars.bat"

Let's take an example assembly, dotNetAssembly.dll, made from the project file above. To disassemble the assembly use this command:

 
ildasm dotNetAssembly.dll /linenum /out:dotNetAssembly.il

This produces the named IL file and will also store any unmanaged resources in a file called dotNetAssembly.res and any managed resources in files with the names that are specified in the assembly metadata. The /linenum option will cause the IL file to include references to the original source lines, assuming debug information is available in a PDB file.

Round Tripping, Step 2: Reassembly

To re-assemble everything back to a .NET assembly you use the IL Assembler, ilasm.exe, which comes as part of the .NET Framework:

 
ilasm /dll dotNetAssembly.il /out:dotNetAssembly.dll /res:dotNetAssembly.res /quiet

Modifying A .NET Assembly Manifest

To expose methods from an assembly you must make some changes to the assembly manifest (found at the top of the IL file before the class declarations) before reassembly. There will be references to other assemblies in the manifest as well as general module information:

 
.assembly extern System.Windows.Forms
{
  .publickeytoken = (B7 7A 5C 56 19 34 E0 89 )                         // .z\V.4..
  .ver 1:0:5000:0
}
.assembly dotNetAssembly
{
  // --- The following custom attribute is added automatically, do not uncomment -------
  //  .custom instance void [mscorlib]System.Diagnostics.DebuggableAttribute::.ctor(bool,
  //                                                                                bool) = ( 01 00 00 01 00 00 ) 
  .hash algorithm 0x00008004
  .ver 0:0:0:0
}
.module dotNetAssembly.dll
// MVID: {B865276C-A90F-4CA2-8AF1-0BF42A04A451}
.imagebase 0x00400000
.subsystem 0x00000002
.file alignment 512
.corflags 0x00000001

The first change is to define a v-table fixup containing as many slots as there are methods to export. In our case we have two methods to export, so the manifest should be changed to this:

 
.assembly extern System.Windows.Forms
{
  .publickeytoken = (B7 7A 5C 56 19 34 E0 89 )                         // .z\V.4..
  .ver 1:0:5000:0
}
.assembly dotNetAssembly
{
  // --- The following custom attribute is added automatically, do not uncomment -------
  //  .custom instance void [mscorlib]System.Diagnostics.DebuggableAttribute::.ctor(bool,
  //                                                                                bool) = ( 01 00 00 01 00 00 ) 
  .hash algorithm 0x00008004
  .ver 0:0:0:0
}
.module dotNetAssembly.dll
// MVID: {B865276C-A90F-4CA2-8AF1-0BF42A04A451}
.imagebase 0x00400000
.subsystem 0x00000002
.file alignment 512
.corflags 0x00000001
.data VT_01 = int32[2]
.vtfixup [2] int32 fromunmanaged at VT_01

Notice that the number of methods is specified twice, once in the integer array (which is data space used for the v-table fixup, each slot being 32-bits in size) and once in the v-table fixup definition (which is defined to contain two slots and is mapped over the integer array).

Before leaving the manifest there is one other change that must be made if you want your assembly to operate on Windows XP and later. By default the .corflags directive, which sets the runtime header flags, specifies a value of 1, which equates to the COMIMAGE_FLAGS_ILONLY flag (defined in the CorHdr.h include file in the .NET Framework SDK). If this flag is set, the XP loader ignores the key section of the assembly file and the fixups are not fixed up. This causes fatal errors when trying to use the assembly exports. The reason this issue occurs on XP is because this was the first version of Windows to incorporate awareness of .NET. The executable file loader subsystem can recognize a .NET binary and cater for it directly. To resolve the problem you must specify the COMIMAGE_FLAGS_32BITREQUIRED flag (whose value is 2) to get the correct behaviour out of the updated loader:

 
.assembly extern System.Windows.Forms
{
  .publickeytoken = (B7 7A 5C 56 19 34 E0 89 )                         // .z\V.4..
  .ver 1:0:5000:0
}
.assembly dotNetAssembly
{
  // --- The following custom attribute is added automatically, do not uncomment -------
  //  .custom instance void [mscorlib]System.Diagnostics.DebuggableAttribute::.ctor(bool,
  //                                                                                bool) = ( 01 00 00 01 00 00 ) 
  .hash algorithm 0x00008004
  .ver 0:0:0:0
}
.module dotNetAssembly.dll
// MVID: {B865276C-A90F-4CA2-8AF1-0BF42A04A451}
.imagebase 0x00400000
.subsystem 0x00000002
.file alignment 512
.corflags 0x00000002
.data VT_01 = int32[2]
.vtfixup [2] int32 fromunmanaged at VT_01

Exporting .NET Methods

Now that we have a v-table fixup for the exported methods, we must get each method to appear as an entry in it. The IL representation of the methods currently looks like this:

 
.method public static void  DoSomething(int32 I) cil managed
{
  // Code size       13 (0xd)
  .maxstack  1
  .line 32:0 'dotNetAssembly.dpr'
  IL_0000:  ldarg.0
  IL_0001:  call       string Borland.Delphi.System.Unit::IntToStr(int32)
  IL_0006:  call       valuetype [System.Windows.Forms]System.Windows.Forms.DialogResult
                         [System.Windows.Forms]System.Windows.Forms.MessageBox::Show(string)
  IL_000b:  pop
  .line 33:0
  IL_000c:  ret
} // end of method Unit::DoSomething
 
.method public static void  DoSomethingElse([in] string Msg) cil managed
{
  // Code size       8 (0x8)
  .maxstack  1
  .line 37:0
  IL_0000:  ldarg.0
  IL_0001:  call       valuetype [System.Windows.Forms]System.Windows.Forms.DialogResult
                         [System.Windows.Forms]System.Windows.Forms.MessageBox::Show(string)
  IL_0006:  pop
  .line 38:0
  IL_0007:  ret
} // end of method Unit::DoSomethingElse

To turn them into unmanaged exports they should be changed to:

 
.method public static void  DoSomething(int32 I) cil managed
{
  // Code size       13 (0xd)
  .maxstack  1
  .line 32:0 'dotNetAssembly.dpr'
  .vtentry 1:1
  .export [1] as DoSomething
  IL_0000:  ldarg.0
  IL_0001:  call       string Borland.Delphi.System.Unit::IntToStr(int32)
  IL_0006:  call       valuetype [System.Windows.Forms]System.Windows.Forms.DialogResult
                         [System.Windows.Forms]System.Windows.Forms.MessageBox::Show(string)
  IL_000b:  pop
  .line 33:0
  IL_000c:  ret
} // end of method Unit::DoSomething
 
.method public static void  DoSomethingElse([in] string Msg) cil managed
{
  // Code size       8 (0x8)
  .maxstack  1
  .line 37:0
  .vtentry 1:2
  .export [2] as DoSomethingElse
  IL_0000:  ldarg.0
  IL_0001:  call       valuetype [System.Windows.Forms]System.Windows.Forms.DialogResult
                         [System.Windows.Forms]System.Windows.Forms.MessageBox::Show(string)
  IL_0006:  pop
  .line 38:0
  IL_0007:  ret
} // end of method Unit::DoSomethingElse

Each method requires a .vtentry directive to link it to the v-table fixup (the red font shows the slot number being specified) and an .export directive to indicate the exported name.

Assembling the file with the appropriate command-line produces:

 
C:\Temp> ilasm /dll dotNetAssembly.il /out:dotNetAssembly.dll /res:dotNetAssembly.res /quiet
Microsoft (R) .NET Framework IL Assembler.Version 1.1.4322.573
Copyright (C) Microsoft Corporation 1998-2002. All rights reserved.
Assembling 'dotNetAssembly.il' , no listing file, to DLL --> 'dotNetAssembly.dll'
Source file is ANSI
 
EmitExportStub: dwVTFSlotRVA=0x00000000
EmitExportStub: dwVTFSlotRVA=0x00000004
Writing PE file
Operation completed successfully

The important information is that two export stubs are emitted. Indeed, using Delphi's TDump utility with the -ee command-line switch (to list the exported routines) proves the point:

 
C:\Temp>tdump -ee dotnetassembly.dll
Turbo Dump  Version 5.0.16.12 Copyright (c) 1988, 2000 Inprise Corporation
                Display of File DOTNETASSEMBLY.DLL
 
EXPORT ord:0001='DoSomething'
EXPORT ord:0002='DoSomethingElse'

Note that ilasm.exe is invoked with the /quiet command-line option which causes the omission of the usual output of every single method listed out as it gets assembled, followed by details of how many members were emitted in the PE file for each class.

These routines can now be called just like any other DLL routine from a Win32 application:

 
unit dotNetAssemblyImport;
 
interface
 
procedure DoSomething(I: Integer); stdcall;
procedure DoSomethingElse(Msg: PChar); stdcall;
 
implementation
 
const
  dotNETAssembly = 'dotNETAssembly.dll';
 
procedure DoSomething(I: Integer); stdcall; external dotNETAssembly;
procedure DoSomethingElse(Msg: PChar); stdcall; external dotNETAssembly;
 
end.

 

The same data marshaling rules apply with Inverse P/Invoke routines as apply to normal P/Invoke routines. Since we didn't specify any marshaling attributes in the declarations of the routines, the String parameter in DoSomethingElse will be marshaled as an ANSI string and so map onto a Win32 PChar (this is taken account of in the Win32 import unit above).

A Maintenance Nightmare?

The main issue developers see with Inverse P/Invoke is the maintenance problem. If you have to modify the compiler-generated assembly through creative round tripping then what happens when you recompile the DLL with your compiler? It would appear you have to go back and manually update the assembly again to export the routines.

Whilst this argument is valid, there is nothing stopping you from writing a utility that automates this post-compilation phase. Such a utility could then be incorporated into the build process and always be executed after the assembly is produced by the compiler.

Such a utility would be best written as a command-line application, however a GUI project that shows the idea accompanies this paper, called mme.dpr (Managed Method Exporter).

Summary

This paper has looked at the mechanisms that facilitate building Windows systems out of Win32 and .NET code. This will continue to be a useful technique whilst .NET is still at an early stage of its life and Win32 dominates in terms of existing systems and developer skills. Indeed, due to the nature of legacy code this may continue long after .NET programming dominates the Windows arena.

The coverage of interoperability mechanism has been intended to be complete enough to get you started without having too many unanswered questions. However it is inevitable in a paper of this size that much information has been omitted. The references below should provide much of the information that could not be fitted into this paper.

References

  1. .NET Interoperability: COM Interop by Brian Long.
    This paper looks at the issues involved in .NET code using Win32 COM objects and also Win32 COM client applications accessing.NET objects, using the COM Interop mechanism.
  2. .NET and COM, The Complete Interoperability Guide by Adam Nathan (of Microsoft), SAMS.
    This covers everything you will need to know about interoperability between .NET and COM, plus lots more you won't ever need.
  3. PINVOKE.NET, managed by Adam Nathan.
    This is an online repository of C# P/Invoke signatures for Win32 APIs.
  4. Delphi for .NET Developerí Guide by Xavier Pacheco.
    There is plenty of additional detail to the subjects covered in this paper in my chapter contributed to this book: Chapter 16, Interoperability Ė COM Interop and the Platform Invocation Service.
  5. Inside Microsoft .NET IL Assembler by Serge Lidin (of Microsoft), Microsoft Press.
    This book describes the CIL (Common Intermediate Language) in detail and is the only text I've seen that shows how to export .NET assembly methods for Win32 clients. The author was responsible for developing the IL Disassembler and Assembler and various other aspects of the .NET Framework.

About Brian Long

Brian Long used to work at Borland UK, performing a number of duties including Technical Support on all the programming tools. Now he is a trouble-shooter, trainer and technical writer on all things .NET as well as a malware (spyware, adware, rootkits etc.) remover.

Besides authoring a Borland Pascal problem-solving book published in 1994 and contributing towards a Delphi for .NET book in 2004, Brian is a regular columnist in The Delphi Magazine and has had numerous articles published in Developer's Review, Computing, Delphi Developer's Journal and EXE Magazine. He was nominated for the Spirit of Delphi 2000 award and was voted Best Speaker at Borland's BorCon 2002 conference in Anaheim, California by the conference delegates.

There are a growing number of conference papers and articles available on Brian's Web site, so feel free to have a browse.