OpenHTF vs pytest for Hardware Testing

OpenHTF and pytest are the two most common Python frameworks engineers weigh up for hardware test automation. They solve different problems. OpenHTF was built by Google for manufacturing test. pytest was built for software testing and adapted by hardware teams. A third option, the TofuPilot Framework, is also purpose-built for hardware and covered below. This guide compares all three with real code so you can pick the right one.

Feature Comparison

The Same Test in Each Framework

Here's a functional test that verifies a PCBA's power rail and firmware version, written in all three.

OpenHTF Version

import openhtf as htffrom openhtf.plugs import BasePlugfrom openhtf.util import unitsfrom tofupilot.openhtf import uploadclass DutPlug(BasePlug):    """Manage DUT connection lifecycle."""    def setUp(self):        self.connected = True  # Replace with real connection    def read_voltage(self) -> float:        return 3.31  # Replace with instrument read    def query_firmware(self) -> str:        return "2.1.0"  # Replace with DUT query    def tearDown(self):        self.connected = False@htf.measures(    htf.Measurement("rail_3v3")    .in_range(3.2, 3.4)    .with_units(units.VOLT)    .doc("3.3V supply rail voltage"),)@htf.PhaseOptions(timeout_s=10)@htf.plug(dut=DutPlug)def test_power(test, dut):    test.measurements.rail_3v3 = dut.read_voltage()@htf.measures(    htf.Measurement("firmware_version")    .equals("2.1.0")    .doc("Expected firmware version string"),)@htf.PhaseOptions(timeout_s=5)@htf.plug(dut=DutPlug)def test_firmware(test, dut):    test.measurements.firmware_version = dut.query_firmware()def main():    test = htf.Test(        test_power,        test_firmware,        procedure_id="FCT-001",        part_number="PCBA-100",    )    test.add_output_callbacks(upload())    test.execute(test_start=lambda: input("Scan serial number: "))if __name__ == "__main__":    main()

Install the OpenHTF extra with pip install "tofupilot[openhtf]". upload() reads TOFUPILOT_API_KEY from the environment. Alternatively, run the same suite with tofupilot run and the CLI streams and uploads it with no callback at all.

pytest Version

import pytestfrom tofupilot import TofuPilotClient@pytest.fixture(scope="session")def tofupilot():    """Initialize TofuPilot client for the test session."""    return TofuPilotClient()@pytest.fixture(scope="session")def serial_number():    """Prompt operator for serial number before tests run."""    return input("Scan serial number: ")@pytest.fixturedef dut():    """Manage DUT connection. Equivalent to an OpenHTF Plug."""    # Setup    connection = {"connected": True}  # Replace with real connection    yield connection    # Teardown    connection["connected"] = Falsedef read_voltage() -> float:    """Read voltage from instrument."""    return 3.31  # Replace with real readdef query_firmware() -> str:    """Query firmware version from DUT."""    return "2.1.0"  # Replace with real querydef test_power_rail(dut):    """Verify 3.3V supply rail is within spec."""    voltage = read_voltage()    assert 3.2 <= voltage <= 3.4, f"3.3V rail out of range: {voltage}V"def test_firmware_version(dut):    """Verify firmware version matches expected."""    version = query_firmware()    assert version == "2.1.0", f"Unexpected firmware: {version}"

TofuPilot Framework Version

The procedure YAML declares the sequence and the limits. Phases are plain functions with no decorators, and the plug is a plain class.

name: PCBA Functional Testversion: 1.0.0unit:  serial_number:    default_value: "PCBA000001"  part_number:    default_value: "PCBA-100"plugs:  - name: dut    python: plugs.dut:Dutmain:  - name: Power    python: phases.power    measurements:      - name: Rail 3V3        unit: V        validators:          - operator: ">="            expected_value: 3.2          - operator: "<="            expected_value: 3.4  - name: Firmware    python: phases.firmware    measurements:      - name: Firmware Version        validators:          - operator: "=="            expected_value: "2.1.0"

def power(measurements, dut):    measurements.rail_3v3 = dut.read_voltage()

def firmware(measurements, dut):    measurements.firmware_version = dut.query_firmware()

class Dut:    def read_voltage(self) -> float:        return 3.31  # Replace with an instrument read    def query_firmware(self) -> str:        return "2.1.0"  # Replace with a DUT query

Run it with tofupilot run ./procedure.yaml. The measurement key is generated from the name in snake_case (Rail 3V3 becomes rail_3v3), and plugs are injected into phases by parameter name. Because the two phases are independent, the engine runs them in parallel.

Key Differences in the Code

When to Use OpenHTF

OpenHTF is the better choice when:

• You already run it. Existing OpenHTF suites run natively on the TofuPilot CLI with no rewrite. If it works, the migration cost isn't worth it.
• You want structured test data in one pure-Python paradigm. Every measurement carries its name, value, limits, units, and pass/fail status, all declared in Python decorators with no separate YAML layer.
• Maturity matters most. Years of production use at scale and a larger body of public examples and patterns than newer options.
• Apache 2.0 fits your legal review. Its explicit patent grant can matter in some procurement contexts.

When to Use pytest

pytest is the better choice when:

• You're doing R&D or validation testing. pytest's flexibility shines when test requirements change frequently. No strict phase ordering, easy to skip or select tests, rich assertion messages.
• Your team knows pytest. Most Python developers already know it. Both hardware frameworks have a learning curve.
• You want a plugin ecosystem. pytest-timeout, pytest-repeat, pytest-html, pytest-cov. Thousands of plugins for every need.
• You're testing firmware/software on hardware. If the "test" is really a software test that happens to run on hardware (flashing firmware, running integration tests on a dev board), pytest is the natural fit.

When to Use the TofuPilot Framework

The TofuPilot Framework is the better choice when:

• Your line is throughput-bound. Parallel phases and multi-slot fixtures test several boards at once on one fixture and cut cycle time. OpenHTF executes sequentially.
• Operators need guided workflows. Text inputs, checklists, images, sliders and progress bars are declared in the procedure and render on the station with no frontend work.
• You validate waveforms or sweeps. Built-in aggregations and per-axis validators check curve shape, not only endpoints.
• You want the test plan separate from the logic. The YAML procedure is readable and diffable independently of the Python phases.
• You are setting up new stations. Deploy-on-push, instant rollback, real-time streaming and offline queueing are built into the CLI.

Using Them Together

Many teams use more than one. pytest for firmware validation and CI, a hardware framework for production functional test:

All three work with TofuPilot. The TofuPilot Framework and OpenHTF both run natively on the CLI; pytest works through the Python SDK.

TofuPilot Integration Comparison

TofuPilot Framework: nothing to add

tofupilot run ./procedure.yaml --upload

Phases, measurements, logs, attachments and the operator UI stream to the dashboard with no code.

OpenHTF: native on the CLI, or two lines

from tofupilot.openhtf import upload# Wrap your test executiontest.add_output_callbacks(upload())test.execute(test_start=lambda: input("Serial: "))# Measurements, limits, units, phases, attachments# all logged automatically.

Or skip the callback entirely and run the suite with tofupilot run.

pytest: Python SDK

from tofupilot import TofuPilotClientclient = TofuPilotClient()# After collecting results, create a runclient.create_run(    procedure_id="FCT-001",    unit_under_test={"serial_number": "SN-001", "part_number": "PCBA-100"},    run_passed=True,    steps=[        {            "name": "test_power_rail",            "step_passed": True,            "measurements": [                {                    "name": "rail_3v3",                    "value": 3.31,                    "unit": "V",                    "lower_limit": 3.2,                    "upper_limit": 3.4,                },            ],        },    ],)

More code, but you get the same analytics in TofuPilot: FPY, Cpk, control charts, failure Pareto, traceability.

Decision Matrix

If you are starting production test from scratch, the TofuPilot Framework is the shortest path: tofupilot run ./procedure.yaml executes on your machine without an account. If you are doing software-style testing on hardware, pytest. If you already have OpenHTF in production, keep it. All three upload to the same dashboard.

For a detailed comparison of the two purpose-built options, see TofuPilot Framework vs OpenHTF.