Calling Functions in Standard DLL Files

InstallShield 2016

Project • This information applies to the following project types:

Basic MSI
InstallScript MSI

Although Windows Installer restricts the parameters that you can pass to a functions in a DLL file written for a custom action, InstallShield offers a solution that enables you to pass any number of parameters to a function and store the return value. Note that the function must use the __stdcall calling convention. Functions with more than one parameter will not work properly if this convention is not used.

You can specify that you are calling a function in a standard DLL file in the Custom Action Wizard’s Action Type panel by selecting Call a function in a standard dynamic-link library for the Type option. The next panel, the Function Definition panel, allows you to specify the function’s parameters and return value.

Note • If you call a function in a standard DLL file that is installed with the product and the action is scheduled as deferred (for the In-Script Execution setting), the DLL file that you are calling must be the component’s key file.

A custom action that calls a function in a standard DLL file stored in the Binary table cannot be scheduled for deferred execution.

If you want a Windows Installer property to be set to the function’s return value, the action must be configured for immediate execution. If you try to specify a return property for an action that is scheduled for deferred, rollback, or commit execution, the return property is ignored at run time.

Understanding the Parameters and Return Value as a Formatted String

When you specify the function signature for a standard DLL file action, that action’s Function Signature setting in the Custom Actions and Sequences view uses the following formatting to display the function’s name, arguments, return type, and return property.

DataType1=[PropertyName1] DllName::FuncName(in DataType2="Value", Direction DataType3=[PropertyName2])

 

The following table describes each portion of the format.

Format for the Arguments of a Function

Argument

Description

DataType1=[PropertyName1]

DataType1 is of type void, STRING, BOOL, NUMBER, HWND, HANDLE, or POINTER. PropertyName1 is the name of a property from the Property Manager view.

If the return type is void, =[PropertyName1] is omitted.

DllName

Specify the name of the DLL file, without the dot or file extension.

FuncName

Enter the name of the function that you are calling.

in DataType2="Value"

This is the format for an argument that is a constant. DataType1 is of type STRING, BOOL, NUMBER, HWND, HANDLE, or POINTER. Value is the data that you want to pass for this argument. The constant’s data type is always preceded by in.

Direction DataType3=[PropertyName2]

This is the format for an argument that references a Windows Installer property’s value. Direction can be in, inout, or out. For a discussion of these property types, see the Function Definition panel under the argument’s source.

DataType3 can be of type STRING, BOOL, NUMBER, HWND, HANDLE, or POINTER. PropertyName2 is the name of a property whose value will be passed to or set by the function, depending on the value of Direction.

Example

For the custom action created in Calling MessageBoxA in an Installation, the Target value reads:

void User32::MessageBoxA(in HWND=0, in STRING=[MESSAGEPROP], in STRING=[CAPTIONPROP], in NUMBER=1)

 

In this example, the data type is void, in HWND=0 represents a numeric value of 0, and MESSAGEPROP is a property that contains a string that will be passed to the function MessageBoxA in User32.dll.