OpenHTF vs pytest for Hardware Testing

OpenHTF and pytest are the two most common Python frameworks for hardware test automation. They solve different problems. OpenHTF was built by Google for manufacturing test. pytest was built for software testing and adapted by hardware teams. This guide compares them side by side with real code so you can pick the right one.

Feature Comparison

Feature	OpenHTF	pytest
Designed for	Manufacturing/production test	Software testing (adapted for hardware)
Test structure	Phases (ordered sequence)	Functions (unordered by default)
Measurements	Built-in: name, value, limits, units	Manual: assert statements or custom fixtures
Serial number input	Built-in (operator prompt)	Manual implementation
Operator UI	Built-in web UI	None (third-party or custom)
Instrument lifecycle	Plugs (auto setup/teardown)	Fixtures (similar, more flexible)
Attachments	Built-in (files, images, logs)	Manual (save to disk or custom plugin)
Test report	Structured protobuf output	JUnit XML, custom plugins
Parallel DUT testing	Limited (single test executor)	Native (pytest-xdist)
Community size	Small (~640 GitHub stars)	Massive (11K+ stars, huge ecosystem)
Plugin ecosystem	Minimal	Thousands of plugins
Learning curve	Steeper (less documentation)	Gentler (extensive documentation)
TofuPilot integration	Native (1 line of code)	Via Python SDK

The Same Test in Both Frameworks

Here's a functional test that verifies a PCBA's power rail and communication interface, written in both frameworks.

OpenHTF Version

comparison/openhtf_test.py

import openhtf as htf
from openhtf.plugs import BasePlug
from openhtf.util import units
from tofupilot.openhtf import TofuPilot


class DutPlug(BasePlug):
    """Manage DUT connection lifecycle."""

    def setUp(self):
        self.connected = True  # Replace with real connection

    def read_voltage(self) -> float:
        return 3.31  # Replace with instrument read

    def query_firmware(self) -> str:
        return "2.1.0"  # Replace with DUT query

    def tearDown(self):
        self.connected = False


@htf.measures(
    htf.Measurement("rail_3v3")
    .in_range(3.2, 3.4)
    .with_units(units.VOLT)
    .doc("3.3V supply rail voltage"),
)
@htf.PhaseOptions(timeout_s=10)
@htf.plug(dut=DutPlug)
def test_power(test, dut):
    test.measurements.rail_3v3 = dut.read_voltage()


@htf.measures(
    htf.Measurement("firmware_version")
    .equals("2.1.0")
    .doc("Expected firmware version string"),
)
@htf.PhaseOptions(timeout_s=5)
@htf.plug(dut=DutPlug)
def test_firmware(test, dut):
    test.measurements.firmware_version = dut.query_firmware()


def main():
    test = htf.Test(
        test_power,
        test_firmware,
        procedure_id="FCT-001",
        part_number="PCBA-100",
    )
    with TofuPilot(test):
        test.execute(test_start=lambda: input("Scan serial number: "))


if __name__ == "__main__":
    main()

pytest Version

comparison/pytest_test.py

import pytest
from tofupilot import TofuPilotClient


@pytest.fixture(scope="session")
def tofupilot():
    """Initialize TofuPilot client for the test session."""
    return TofuPilotClient()


@pytest.fixture(scope="session")
def serial_number():
    """Prompt operator for serial number before tests run."""
    return input("Scan serial number: ")


@pytest.fixture
def dut():
    """Manage DUT connection. Equivalent to an OpenHTF Plug."""
    # Setup
    connection = {"connected": True}  # Replace with real connection
    yield connection
    # Teardown
    connection["connected"] = False


def read_voltage() -> float:
    """Read voltage from instrument."""
    return 3.31  # Replace with real read


def query_firmware() -> str:
    """Query firmware version from DUT."""
    return "2.1.0"  # Replace with real query


def test_power_rail(dut):
    """Verify 3.3V supply rail is within spec."""
    voltage = read_voltage()
    assert 3.2 <= voltage <= 3.4, f"3.3V rail out of range: {voltage}V"


def test_firmware_version(dut):
    """Verify firmware version matches expected."""
    version = query_firmware()
    assert version == "2.1.0", f"Unexpected firmware: {version}"

Key Differences in the Code

Aspect	OpenHTF	pytest
Measurements	Declarative with `@htf.measures`. Name, limits, units defined upfront.	Implicit via `assert`. No structured metadata.
Serial number	Built into `test.execute()`.	Custom fixture or input call.
DUT lifecycle	Plug class with `setUp`/`tearDown`. Injected via `@htf.plug()` decorator.	Fixture with `yield`. Manually passed.
Limits	`.in_range(3.2, 3.4)` stored as structured data.	`assert 3.2 <= v <= 3.4` is code, not data.
Output	Structured protobuf with all metadata.	Pass/fail only (unless you add custom reporting).
Timeouts	`@htf.PhaseOptions(timeout_s=10)`	`@pytest.mark.timeout(10)` (requires plugin)

When to Use OpenHTF

OpenHTF is the better choice when:

You're running production tests. The built-in measurement model (name, value, limits, units) maps directly to how manufacturing test data needs to be stored and analyzed. You don't need to build this yourself.
You need operator interaction. OpenHTF has a built-in web UI for operators to scan serial numbers, see test progress, and view results. With pytest, you'd build this from scratch.
You want structured test data. Every measurement in OpenHTF carries its name, value, limits, units, and pass/fail status. This feeds directly into TofuPilot for FPY, Cpk, and control chart analysis. With pytest, you get pass/fail per test function, nothing more.
Your team already uses OpenHTF. If you have existing OpenHTF tests, stay with it. The migration cost isn't worth the switch.

When to Use pytest

pytest is the better choice when:

You're doing R&D or validation testing. pytest's flexibility shines when test requirements change frequently. No strict phase ordering, easy to skip or select tests, rich assertion messages.
You need parallel DUT testing. pytest-xdist runs tests in parallel natively. OpenHTF's executor is single-threaded.
Your team knows pytest. Most Python developers already know pytest. OpenHTF has a learning curve and sparse documentation.
You want a plugin ecosystem. pytest-timeout, pytest-repeat, pytest-html, pytest-cov. Thousands of plugins for every need. OpenHTF has almost none.
You're testing firmware/software on hardware. If the "test" is really a software test that happens to run on hardware (flashing firmware, running integration tests on a dev board), pytest is the natural fit.

Using Both Together

Some teams use both. pytest for firmware validation and CI, OpenHTF for production functional test. This works well when the test requirements are genuinely different:

Test Stage	Framework	Why
Firmware CI	pytest	Runs in CI pipeline, parallel execution, software-style testing
EVT/DVT validation	pytest	Flexible, exploratory, requirements change frequently
PVT/Production FCT	OpenHTF	Structured measurements, operator UI, production data logging
Incoming inspection	OpenHTF	Repeatable, operator-driven, needs traceability

Both frameworks work with TofuPilot. OpenHTF has native integration (one line). pytest works through the Python SDK (a few more lines, same data).

TofuPilot Integration Comparison

OpenHTF: One line

comparison/tofupilot_openhtf.py

from tofupilot.openhtf import TofuPilot

# Wrap your test execution
with TofuPilot(test):
    test.execute(test_start=lambda: input("Serial: "))
# Measurements, limits, units, phases, attachments
# all logged automatically.

pytest: Python SDK

comparison/tofupilot_pytest.py

from tofupilot import TofuPilotClient

client = TofuPilotClient()

# After collecting results, create a run
client.create_run(
    procedure_id="FCT-001",
    unit_under_test={"serial_number": "SN-001", "part_number": "PCBA-100"},
    run_passed=True,
    steps=[
        {
            "name": "test_power_rail",
            "step_passed": True,
            "measurements": [
                {
                    "name": "rail_3v3",
                    "value": 3.31,
                    "unit": "V",
                    "lower_limit": 3.2,
                    "upper_limit": 3.4,
                },
            ],
        },
    ],
)

More code, but you get the same analytics in TofuPilot: FPY, Cpk, control charts, failure Pareto, traceability.

Decision Matrix

Answer these questions:

Question	If yes →	If no →
Are you running production/manufacturing tests?	OpenHTF	pytest
Do operators interact with the test station?	OpenHTF	pytest
Do you need structured measurement data (limits, units)?	OpenHTF	pytest
Do you need parallel DUT testing?	pytest	Either
Is your team new to both frameworks?	pytest (easier to learn)	N/A
Do you need a rich plugin ecosystem?	pytest	Either
Are you doing firmware CI testing?	pytest	N/A

If you answered "yes" to the first three questions, start with OpenHTF. For everything else, pytest is the safer bet. Both work with TofuPilot.