OpenHTF vs NI TestStand for Manufacturing Test

OpenHTF is a free, open-source test framework from Google. NI TestStand is a commercial test sequencer from NI (Emerson) at $4,310/seat/year. Both run manufacturing tests with measurements, limits, and sequencing. They differ in cost, platform support, database integration, and how you manage test data. This guide compares them side by side with real code and concrete tradeoffs.

Feature Comparison

Cost Analysis

TestStand also requires Windows, which means Windows licenses for every test station. OpenHTF runs on Linux, which is free and more stable for long-running production stations.

The Same Test in Both Frameworks

A simple functional test: measure a 3.3V rail, check it's within 3.2V to 3.4V.

OpenHTF + TofuPilot

import openhtf as htf
from openhtf.util import units
from tofupilot.openhtf import TofuPilot


@htf.measures(
    htf.Measurement("rail_3v3")
    .in_range(3.2, 3.4)
    .with_units(units.VOLT),
)
def test_power_rail(test):
    voltage = 5.02  # Replace with instrument read
    test.measurements.rail_3v3 = voltage


def main():
    test = htf.Test(
        test_power_rail,
        procedure_id="FCT-001",
        part_number="PCBA-100",
    )
    with TofuPilot(test):
        test.execute(test_start=lambda: input("Scan serial: "))


if __name__ == "__main__":
    main()

Measurements are structured data: name, value, limits, units. TofuPilot integration is one with statement. The operator gets a serial number prompt automatically.

NI TestStand

In TestStand, you create a sequence file (.seq) in the Sequence Editor. Add a "Numeric Limit Test" step, set the test expression to your instrument read, configure Low Limit = 3.2 and High Limit = 3.4. Connect the step to a Code Module (LabVIEW VI, DLL, or .NET assembly) that talks to the instrument.

The test logic lives in the Code Module. The sequencing, limits, and reporting live in the .seq file. You can't see both in a single text file, and you can't diff the .seq file in Git.

Concept Mapping

Every TestStand concept has a direct OpenHTF equivalent.

Instrument Drivers

TestStand uses NI VISA and IVI drivers. OpenHTF uses PyVISA, which talks to the same instruments through the same VISA layer. If your instrument works with TestStand, it works with PyVISA.

NI publishes official Python packages for most of their hardware. You don't lose instrument support by switching to Python.

OpenHTF Plug for an Instrument

TestStand wraps instruments in Code Modules. OpenHTF wraps them in Plugs, which have automatic lifecycle management (setUp/tearDown).

import pyvisa
from openhtf.plugs import BasePlug
import openhtf as htf
from openhtf.util import units
from tofupilot.openhtf import TofuPilot


class MultimeterPlug(BasePlug):
    """Wraps a SCPI multimeter connection."""

    def setUp(self):
        rm = pyvisa.ResourceManager()
        self.instr = rm.open_resource("TCPIP::192.168.1.100::INSTR")
        self.instr.timeout = 5000

    def measure_voltage(self, channel=1):
        self.instr.write(f":CONF:VOLT:DC AUTO,(@{channel})")
        self.instr.write(":INIT")
        return float(self.instr.query(":FETCH?"))

    def tearDown(self):
        self.instr.close()


@htf.measures(
    htf.Measurement("rail_3v3")
    .in_range(3.2, 3.4)
    .with_units(units.VOLT),
)
@htf.plug(dmm=MultimeterPlug)
def measure_power_rail(test, dmm):
    test.measurements.rail_3v3 = dmm.measure_voltage(channel=1)


def main():
    test = htf.Test(measure_power_rail, procedure_id="FCT-001", part_number="PCBA-100")
    with TofuPilot(test):
        test.execute(test_start=lambda: input("Scan serial: "))


if __name__ == "__main__":
    main()

Sharing Data Between Steps

TestStand uses FileGlobals and StationGlobals to pass data between steps. OpenHTF uses plug instance attributes. The plug persists for the entire test execution.

from openhtf.plugs import BasePlug
import openhtf as htf
from openhtf.util import units


class SharedState(BasePlug):
    """Replaces TestStand FileGlobals and StationGlobals."""

    def setUp(self):
        self.cal_offset = 0.0
        self.firmware_version = ""

    def tearDown(self):
        pass


@htf.plug(state=SharedState)
def calibrate(test, state):
    state.cal_offset = 0.023


@htf.measures(
    htf.Measurement("corrected_voltage")
    .in_range(3.2, 3.4)
    .with_units(units.VOLT),
)
@htf.plug(state=SharedState)
def measure_corrected(test, state):
    raw = 3.323  # Replace with instrument read
    test.measurements.corrected_voltage = raw - state.cal_offset

Database and Test Data

This is where the two frameworks differ most.

TestStand Database Logging

TestStand's built-in database logger writes results to SQL Server, Oracle, or Access using a fixed schema. The core tables are UUT_RESULT, STEP_RESULT, and PROP_RESULT, with additional tables for each property type (PROP_NUMERICLIMIT, PROP_STRINGVALUE, etc.).

Querying this schema requires multi-level JOINs:

A simple query to get one unit's measurements requires 5-6 table JOINs. Adding custom metadata means modifying the Process Model's database schema mapping, which is fragile and hard to maintain across TestStand versions.

TestStand doesn't include analytics. FPY, Cpk, control charts, and failure Pareto all require custom SQL queries or a third-party tool like WATS.

OpenHTF + TofuPilot

OpenHTF doesn't have built-in database logging. TofuPilot handles it. You add one with TofuPilot(test): line and every run is stored with structured measurements, limits, units, serial numbers, and metadata.

No database to provision, no schema to maintain, no SQL to write. TofuPilot tracks everything automatically. Open the Analytics tab to see FPY, Cpk, and failure analysis for any procedure.

Version Control and Git

TestStand sequence files (.seq) are binary. You can store them in Git, but you can't diff them, review changes in a pull request, or merge branches.

NI added a Git pane in TestStand 2025 Q2. It lets you commit, pull, and push .seq files from within the Sequence Editor. But it's file-level tracking, not content-level diffing. You can see that a file changed, not what changed inside it. NI provides a separate "Diff and Merge Utility" that can compare two .seq files visually, but it doesn't integrate with pull request workflows.

OpenHTF tests are plain Python files. Standard Git workflows apply.

What a Python Test Diff Looks Like

When you change a measurement limit in an OpenHTF test, the pull request shows exactly what changed:

@htf.measures(
     htf.Measurement("rail_3v3")
-    .in_range(3.2, 3.4)
+    .in_range(3.1, 3.5)
     .with_units(units.VOLT),
 )

Reviewers see the old limit, the new limit, and the context. In TestStand, the same change is invisible inside a binary .seq file.

CI/CD Integration

NI updated TestStand's license agreement in 2025 Q2 to allow CI/CD usage without extra cost (if you have at least one active development license). But running TestStand in CI still requires a Windows runner with TestStand installed, and the sequence files can't be linted or statically analyzed.

OpenHTF tests are Python. They run in any CI system with a Python environment.

# Lint and type-check OpenHTF test scripts in CI
name: Test Script CI
on: [push, pull_request]
jobs:
  lint:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-python@v5
        with:
          python-version: "3.12"
      - run: pip install ruff mypy openhtf
      - run: ruff check tests/
      - run: mypy tests/

You can catch measurement naming issues, import errors, and type problems before test scripts reach the production floor.

When to Use Each

Migration Path

If you're on TestStand and considering a move, the typical migration takes 4-8 weeks per test procedure. Most teams run both systems in parallel during the transition.

1. Set up Python + OpenHTF alongside TestStand on one station.
2. Convert one test procedure (the simplest one). Run both versions on the same DUTs.
3. Validate measurements match between systems.
4. Move to the next procedure once results are confirmed.
5. Decommission TestStand when all procedures are converted.

The biggest risk is rushing. Convert one procedure at a time. Parallel operation is your safety net.