HardPy vs TofuPilot : comparatif d'exécutifs

HardPy and the TofuPilot Framework are both open-source Python tools for running hardware tests with an operator in front of the station. HardPy is a pytest plugin from Everypin that adds a browser operator panel and stores results in CouchDB. The TofuPilot Framework is a standalone test framework with a Rust execution engine, a YAML procedure and Python phases, connected to the TofuPilot dashboard for analytics and station deployment. This guide compares the two on authoring, operator UI, data, analytics and deployment.

Facts about HardPy were checked against its repository, PyPI and documentation in November 2026 (version 0.25.2).

Overview

Feature Comparison

The Same Test in Both

A supply rail check: measure the 3.3 V rail of a board and judge it against 3.2 V to 3.4 V.

HardPy Version

HardPy tests are pytest functions. A measurement carries its value, its comparison and its limits, and is attached to the current test case.

# HardPy test case: measure the rail and record it with its limits.import pytestfrom hardpy import ComparisonOperation as CompOp, NumericMeasurement, set_case_measurement@pytest.mark.case_name("3.3 V rail")def test_rail_3v3(dmm):    meas = NumericMeasurement(        value=dmm.read_voltage(),        unit="V",        operation=CompOp.GELE,        lower_limit=3.2,        upper_limit=3.4,    )    set_case_measurement(meas)    assert meas.result

The dmm fixture comes from a conftest.py that opens the instrument. hardpy run starts the operator panel on localhost:8000 and runs the tests from it.

TofuPilot Framework Version

The limits live in the procedure, as data. The phase only reads the instrument and assigns the value.

name: Power Checkversion: 1.0.0plugs:  - name: Multimeter    key: dmm    python: plugs.dmm:Multimetermain:  - name: Rail 3V3    key: rail_3v3    python: phases.rail_3v3    measurements:      - name: Rail 3V3        key: rail_3v3        unit: V        validators:          - {operator: ">=", expected_value: 3.2}          - {operator: "<=", expected_value: 3.4}

# Plugs and measurements are injected by parameter name.def rail_3v3(measurements, dmm):    measurements.rail_3v3 = dmm.read_voltage()

tofupilot run executes the procedure locally without an account. Connected to a workspace, the same run streams to the dashboard.

Test Authoring

HardPy keeps everything in pytest. Engineers who already write pytest suites reuse fixtures, markers and plugins, and the limits sit next to the code that measures. The cost is that the test plan is spread across Python files: changing a limit is a code change.

The TofuPilot Framework splits the plan from the logic. procedure.yaml holds the sequence, the limits, the units and the operator components; the phases are plain Python functions. A limit change is a one-line diff in the YAML, reviewable by someone who does not read Python.

Operator UI

Both ship an operator interface in the browser.

• HardPy serves a panel with the test hierarchy, live status, dialog boxes, images, data entry and confirmations. Because the panel is client-server, the operator can use a tablet while the PC runs the test.
• TofuPilot Framework generates the station kiosk from the procedure. Text, number, switch, radio, select, image and progress components are declared in YAML and can be bound directly to measurements or to the unit serial number.

Result Storage and Analytics

This is the largest difference.

HardPy stores each run in a local CouchDB started with Docker Compose, or in JSON files. The open-source library has no analytics: yield, failure Pareto, traceability and fleet monitoring come from StandCloud, Everypin's hosted service, priced per device tested. A self-hosted StandCloud is not documented.

The TofuPilot Framework queues each run on the station and uploads it to the TofuPilot dashboard, which provides yield, first pass yield, control charts, Cpk, measurement history per unit and serial number traceability. The dashboard is available hosted or self-hosted.

Stations and Deployment

HardPy documents running several instances on one PC, each with its own port and database document. It does not document how a new test version reaches a station; that is left to your own process (copying files, Git pull, a package).

The TofuPilot CLI turns a station into a managed target. A procedure is deployed from a Git repository on push, every station runs the same version, and a deployment can be rolled back instantly from the dashboard.

When to Use HardPy

• Your team writes pytest already and wants an operator panel on top with minimal new concepts.
• One or a few benches, where local CouchDB storage is enough.
• Per-device StandCloud pricing fits your volume.
• GPL-3.0 is acceptable for your codebase.

When to Use the TofuPilot Framework

• Several stations that must run the same procedure version, deployed and rolled back centrally.
• Fixtures holding several units, or phases that can run in parallel to cut cycle time.
• Waveforms and sweeps judged on their shape, not only their endpoints.
• Yield and process analytics without a separate service, hosted or self-hosted.
• An MIT license for the test framework.

Decision Matrix

For the wider landscape, see Open Source Test Executive Alternatives and TofuPilot Framework vs OpenHTF.