Call a 32-bit DLL from a TofuPilot Plug

A test station running the TofuPilot CLI gets a 64-bit Python, and a 64-bit process cannot load a 32-bit DLL. Learn how to keep a 32-bit vendor library in service by hosting it in its own 32-bit process with msl-loadlib, and exposing it to your phases as an ordinary plug.

Why a 64-bit Process Cannot Load a 32-bit DLL

A DLL is loaded into the memory space of the process that calls it. Pointers are 8 bytes in a 64-bit process and 4 bytes in a 32-bit one, so the two cannot share an address space. Windows refuses the load before any code runs:

OSError: [WinError 193] %1 is not a valid Win32 application

The station interpreter is 64-bit by construction. The CLI provisions each procedure's virtual environment with uv venv --python 3.12, which takes the native architecture of the machine, and no setting changes it. Old vendor libraries, in-house DLLs whose source is lost and shared libraries built with an old LabVIEW are the usual 32-bit survivors.

Two ways out exist:

This guide does the second. The isolation is the same idea as an optocoupler between two voltage domains: signals cross, wires do not.

Prerequisites

• A Windows test station with the TofuPilot CLI installed
• The 32-bit DLL, with its header file or its documentation, so you know each function prototype and the calling convention (__cdecl or __stdcall)
• msl-loadlib 1.1 or later, which ships a prebuilt 32-bit server for Windows, so no 32-bit Python needs to be installed on the station

Step 1: Add msl-loadlib to the Procedure Dependencies

The CLI installs the requirements of a procedure into its virtual environment on tofupilot run and on every deployment:

msl-loadlib>=1.1

msl-loadlib is a thin wrapper around ctypes. The difference is where the ctypes code runs: inside a 32-bit server process that the library starts for you, instead of inside your 64-bit Python.

Step 2: Move the ctypes Code into a 32-bit Server Class

The server module is imported by the 32-bit server, never by the station Python. Put every ctypes call in it, and every buffer the DLL writes into. The example wraps a fictional I2C library with four functions:

import ctypesfrom msl.loadlib import Server32class BusBridgeServer(Server32):    def __init__(self, host, port, **kwargs):        # kwargs come from the Client64 constructor. msl-loadlib turns        # every value into a string, so convert them back here.        dll_path = kwargs["dll_path"]        device_index = int(kwargs.get("device_index", "0"))        # "cdll" for a __cdecl DLL, "windll" for a __stdcall one.        super().__init__(dll_path, "cdll", host, port)        lib = self.lib        lib.BB_Open.argtypes = [ctypes.c_int]        lib.BB_Open.restype = ctypes.c_int        lib.BB_I2CWrite.argtypes = [            ctypes.c_int,            ctypes.c_int,            ctypes.POINTER(ctypes.c_ubyte),            ctypes.c_int,        ]        lib.BB_I2CWrite.restype = ctypes.c_int        lib.BB_I2CRead.argtypes = [            ctypes.c_int,            ctypes.c_int,            ctypes.POINTER(ctypes.c_ubyte),            ctypes.c_int,        ]        lib.BB_I2CRead.restype = ctypes.c_int        lib.BB_Close.argtypes = [ctypes.c_int]        lib.BB_Close.restype = None        self._handle = lib.BB_Open(device_index)        if self._handle < 0:            raise RuntimeError(f"BB_Open({device_index}) failed with code {self._handle}")    def i2c_write(self, address, data):        buf = (ctypes.c_ubyte * len(data))(*data)        rc = self.lib.BB_I2CWrite(self._handle, address, buf, len(data))        if rc != 0:            raise RuntimeError(f"BB_I2CWrite(0x{address:02X}) failed with code {rc}")    def i2c_read(self, address, length):        buf = (ctypes.c_ubyte * length)()        count = self.lib.BB_I2CRead(self._handle, address, buf, length)        if count < 0:            raise RuntimeError(f"BB_I2CRead(0x{address:02X}) failed with code {count}")        # A list of ints crosses the process boundary. The buffer does not.        return list(buf[:count])    def close(self):        if self._handle >= 0:            self.lib.BB_Close(self._handle)            self._handle = -1

Three rules apply to every method of the server class:

• Arguments and return values are plain Python values: numbers, strings, lists, dicts. They are pickled across the two processes.
• Pointers never cross. A buffer the DLL fills is allocated in the method, read in the method, and returned as a list.
• The library handle is self.lib. It is a ctypes.CDLL for cdll and a ctypes.WinDLL for windll. Set argtypes and restype once in __init__, as you would with plain ctypes.

An exception raised in a server method does not cross as itself. The client raises msl.loadlib.exceptions.Server32Error, whose message carries the server-side traceback, including the text you wrote. The phase sees one exception with a readable cause, which is what a failing phase needs.

Step 3: Write the Plug as a 64-bit Client

The plug is the class the engine instantiates. It inherits from Client64, starts the 32-bit server in its constructor, and forwards each method with request32:

import osfrom msl.loadlib import Client64SERVER_MODULE = os.path.join(os.path.dirname(os.path.abspath(__file__)), "busbridge_server32.py")class BusBridge(Client64):    def __init__(self, dll_path, device_index=0):        # Starts the 32-bit server (a few seconds) and imports        # SERVER_MODULE inside it. The extra keyword arguments are        # handed to BusBridgeServer.__init__ as strings.        super().__init__(SERVER_MODULE, dll_path=dll_path, device_index=device_index)        self.dll_path = dll_path    def i2c_write(self, address, data):        self.request32("i2c_write", address, list(data))    def i2c_read(self, address, length):        return self.request32("i2c_read", address, length)    def __del__(self):        # The engine calls __del__ when the plug is destroyed. Without        # the shutdown, a 32-bit server process would outlive each run.        try:            self.request32("close")        except Exception:            pass        try:            self.shutdown_server32()        except AttributeError:            pass  # the server never started, there is nothing to stop

request32 takes the name of a server method and its positional arguments, and returns what the method returned. The keyword arguments passed to Client64.__init__ reach the server constructor, with one caveat: host, port, timeout, rpc_timeout, server32_dir, append_sys_path, append_environ_path and add_dll_directory are msl-loadlib's own parameters, so name yours differently.

The engine guarantees the __del__ call when the plug is destroyed. The method shuts down the 32-bit server, so no orphan process is left between runs.

Step 4: Declare the Plug and Use It in a Phase

name: EEPROM readback through a 32-bit bus bridge DLLversion: 0.1.0plugs:  - name: Bus Bridge    key: bus_bridge    description: 32-bit vendor DLL hosted in a msl-loadlib 32-bit server    python: plugs.busbridge:BusBridge    scope: station    config:      dll_path: "C:\\BusBridge\\busbridge.dll"      device_index: 0main:  - name: EEPROM header readback    python: phases.i2c_readback    measurements:      - name: eeprom_header_byte0        validators:          - operator: "=="            expected_value: 165

scope: station keeps one instance alive for the life of the station process, so the server start (two to three seconds measured on a Windows runner) and the BB_Open call are paid once, not on every run. The config block is passed to the plug constructor as keyword arguments, which is where the DLL path belongs: a station-specific path has no place in Python code.

The phase receives the plug by its key and calls it like any plug:

EEPROM_ADDRESS = 0x50def i2c_readback(bus_bridge, measurements, log):    # Plug arguments are positional and JSON-serializable: ints and    # lists of ints here, never bytes objects or ctypes buffers.    bus_bridge.i2c_write(EEPROM_ADDRESS, [0x00, 0x00])    header = bus_bridge.i2c_read(EEPROM_ADDRESS, 4)    log.info(f"EEPROM header: {[hex(b) for b in header]}")    measurements.eeprom_header_byte0 = header[0]

Two process boundaries sit between the phase and the DLL: the engine's own plug process, then the 32-bit server. Both carry the same kind of values, so the rule is the same on both sides. Pass numbers, strings and lists, never buffers or handles.

Step 5: Reuse Existing 32-bit Python Modules Without Rewriting Them

If the ctypes code already exists as Python modules written against a 32-bit Python, you do not have to move it into the server class. The server class can import those modules unchanged and delegate to them:

import sysfrom msl.loadlib import Server32# Drop the 64-bit site-packages the client passes along, so the import# below cannot pick up a 64-bit build of numpy or comtypes.Server32.remove_site_packages_64bit()import legacy_busbridge  # the existing module, untouchedclass BusBridgeServer(Server32):    def __init__(self, host, port, **kwargs):        super().__init__(kwargs["dll_path"], "cdll", host, port)        self._dev = legacy_busbridge.BusBridge(int(kwargs["device_index"]))    def i2c_read(self, address, length):        return list(self._dev.read(address, length))

Two things change compared to the first version:

1. The server must find the legacy modules. Pass their directory with append_sys_path in the Client64 constructor.
2. The server must contain their dependencies. The prebuilt server ships with the standard library only. If the legacy modules import numpy, pyserial or comtypes, build a custom server once from the 32-bit Python environment that already runs them:

# In the existing 32-bit Python environment (3.8 or later),# run from the directory that holds the legacy modulespip install msl-loadlib pyinstallerfreeze32 --imports numpy serial legacy_busbridge --dest server32-custom

The command writes a server32-windows.exe into server32-custom. Every module named after --imports is frozen into it, the legacy module included, so append_sys_path is no longer needed for them. Point the plug at that directory with the server32_dir argument of Client64.__init__, and the legacy code runs as it always did, inside its own interpreter. One trap: freeze32 reports a module it cannot import but still exits with status 0, so check that the executable exists before shipping it.

Keep Large Data on the 32-bit Side

Each request32 call pickles its arguments and its result through a local socket. A few hundred bytes cost nothing. A long acquisition does: a one-million-sample waveform in 16-bit is 2 MB per call, and a phase that pulls fifty of them to average a noise figure spends more time copying than measuring.

Move the computation to the data instead. Compute the measurement in the server method and return the numbers:

def rms_noise(self, acquisitions):    """Acquire `acquisitions` waveforms and return RMS noise, computed here."""    import math    total = 0.0    count = 0    for _ in range(acquisitions):        buf = (ctypes.c_int16 * self._samples)()        self.lib.DAQ_Acquire(self._handle, buf, self._samples)        mean = sum(buf) / self._samples        total += sum((s - mean) ** 2 for s in buf)        count += self._samples    return math.sqrt(total / count) * self._volts_per_lsb

The phase receives one float. The samples never leave the 32-bit process.

Common Pitfalls

Passing a pointer or a ctypes object across request32. The call fails to pickle, or worse, the 32-bit side reads a 64-bit address. Allocate in the server method, return values.

Using the wrong calling convention. A __stdcall DLL loaded as cdll corrupts the stack on the first call. The header says which, and windll is the fix.

Forgetting that server kwargs are strings. device_index=0 arrives as "0". Convert in the server constructor.

Importing numpy in the server without a custom build. The prebuilt server has no numpy, and the 64-bit site-packages it inherits holds the wrong one. Call remove_site_packages_64bit() and freeze a server that contains the packages you need.

Declaring the plug with scope: slot on a shared bus. Several slots would start several 32-bit servers against one DLL and one physical interface. Use station or execution so calls serialize through one instance.

Dropping the shutdown_server32() call. Each run leaves a 32-bit process behind. Keep it in __del__.

Sending whole acquisitions through the boundary. Compute in the server, return numbers.

Key Points

• A 64-bit Python cannot load a 32-bit DLL, and the station interpreter is 64-bit with no override.
• msl-loadlib runs your ctypes code in a prebuilt 32-bit server and gives the 64-bit side a client class.
• The plug is the client. Phases see an ordinary plug with ordinary methods.
• Pointers and buffers stay in the server. Only numbers, strings, lists and dicts cross.
• Existing 32-bit Python modules can be hosted unchanged, with a custom frozen server when they need packages.
• Keep heavy data on the 32-bit side and return the measurement, not the raw samples.