Log HIL Results for Regression Analysis

HIL tests generate a lot of data: measurements, waveform captures, firmware versions, environmental conditions. Without structured logging, spotting regressions across firmware releases turns into a spreadsheet archaeology project. This guide shows how to tag, attach, and query HIL results in TofuPilot so regressions surface automatically.

Prerequisites

Python 3.8+
OpenHTF installed (pip install openhtf)
TofuPilot Python client (pip install tofupilot)
A working HIL test (see How to Set Up HIL Testing for Embedded Systems with TofuPilot)

Step 1: Tag Runs with Firmware Version and Hardware Revision

Every HIL test run should carry the firmware version and hardware revision of the DUT. TofuPilot stores these as structured metadata you can filter and query later.

Query the firmware version from the DUT at the start of the test, then pass it to TofuPilot through measurements.

hil_regression/main.py

import openhtf as htf
from tofupilot.openhtf import TofuPilot

from hil_regression.phases import (
    read_firmware_info,
    test_adc_accuracy,
    test_pwm_output,
    test_sleep_current,
)


def main():
    test = htf.Test(
        read_firmware_info,
        test_adc_accuracy,
        test_pwm_output,
        test_sleep_current,
    )

    with TofuPilot(test):
        test.execute(test_start=lambda: input("Scan DUT serial number: "))


if __name__ == "__main__":
    main()

The firmware version and hardware revision get recorded as measurements, making them searchable across all runs.

hil_regression/phases.py

import openhtf as htf
from hil_regression.serial_plug import SerialCommandPlug


@htf.plug(serial=SerialCommandPlug)
@htf.measures(
    htf.Measurement("firmware_version"),
    htf.Measurement("hardware_revision"),
    htf.Measurement("bootloader_version"),
)
def read_firmware_info(test, serial):
    """Record firmware and hardware identifiers for traceability."""
    test.measurements.firmware_version = serial.send_command("VERSION")
    test.measurements.hardware_revision = serial.send_command("HWREV")
    test.measurements.bootloader_version = serial.send_command("BLVER")

This gives you three filterable fields on every run. When FPY drops after a firmware update, filter by firmware_version in TofuPilot's dashboard to isolate the change.

Step 2: Attach Waveform and Log Files

HIL tests often capture oscilloscope waveforms, logic analyzer traces, or DUT console logs. Attach these to the test run so they're available for post-mortem analysis without digging through file shares.

hil_regression/capture.py

import csv
import time
import openhtf as htf
from openhtf.plugs import BasePlug


class WaveformCapturePlug(BasePlug):
    """Captures analog samples and saves to CSV for attachment."""

    def setUp(self):
        pass

    def capture_waveform(self, adc, channel: int, duration_s: float, sample_rate_hz: int) -> str:
        """Sample an ADC channel and write results to a CSV file."""
        filepath = f"/tmp/waveform_ch{channel}_{int(time.time())}.csv"
        samples = []
        interval = 1.0 / sample_rate_hz

        for i in range(int(duration_s * sample_rate_hz)):
            voltage = adc.read_voltage(channel)
            samples.append((i * interval, voltage))
            time.sleep(interval)

        with open(filepath, "w", newline="") as f:
            writer = csv.writer(f)
            writer.writerow(["time_s", "voltage_v"])
            writer.writerows(samples)

        return filepath

    def tearDown(self):
        pass

Then attach the file in your test phase:

hil_regression/phases.py

import openhtf as htf
from openhtf.util import units
from hil_regression.capture import WaveformCapturePlug
from hil_regression.analog_plug import AnalogInputPlug
from hil_regression.serial_plug import SerialCommandPlug


@htf.plug(serial=SerialCommandPlug, adc=AnalogInputPlug, capture=WaveformCapturePlug)
@htf.measures(
    htf.Measurement("pwm_voltage_mean").in_range(minimum=1.6, maximum=1.7).with_units(units.VOLT),
)
def test_pwm_output(test, serial, adc, capture):
    """Command 50% PWM and capture the output waveform."""
    serial.send_command("PWM SET 50")

    # Capture 1 second of data at 1 kHz
    waveform_path = capture.capture_waveform(adc, channel=3, duration_s=1.0, sample_rate_hz=1000)

    # Attach the waveform CSV to the test run
    test.attach("pwm_waveform", waveform_path, "text/csv")

    # Also record the mean voltage as a measurement
    import csv
    with open(waveform_path) as f:
        reader = csv.DictReader(f)
        voltages = [float(row["voltage_v"]) for row in reader]
    test.measurements.pwm_voltage_mean = sum(voltages) / len(voltages)

Attachments show up in TofuPilot's run detail view. You can download them later for offline analysis or comparison across firmware versions.

Step 3: Use Sub-Units for Multi-Board HIL Setups

Many products contain multiple boards (main CPU board, power board, sensor board) that get tested together in a HIL fixture. TofuPilot's sub-units let you track each board independently while keeping them linked to the parent assembly.

hil_regression/multi_board.py

import openhtf as htf
from tofupilot.openhtf import TofuPilot

from hil_regression.phases import (
    test_cpu_board,
    test_power_board,
    test_sensor_board,
)


def main():
    test = htf.Test(
        test_cpu_board,
        test_power_board,
        test_sensor_board,
    )

    with TofuPilot(
        test,
        sub_units=[
            {"serial_number": "CPU-BRD-0042", "part_number": "PCB-CPU-R3"},
            {"serial_number": "PWR-BRD-0019", "part_number": "PCB-PWR-R2"},
            {"serial_number": "SNS-BRD-0088", "part_number": "PCB-SNS-R1"},
        ],
    ):
        test.execute(test_start=lambda: input("Scan assembly serial number: "))


if __name__ == "__main__":
    main()

Each sub-unit gets its own traceability record in TofuPilot. If the sensor board starts failing after a hardware revision change, you can filter by PCB-SNS-R1 and see exactly when failures started.

hil_regression/board_phases.py

import openhtf as htf
from openhtf.util import units
from hil_regression.analog_plug import AnalogInputPlug
from hil_regression.serial_plug import SerialCommandPlug


@htf.plug(serial=SerialCommandPlug)
@htf.measures(
    htf.Measurement("cpu_clock_mhz").in_range(minimum=79, maximum=81).with_units(units.HERTZ),
    htf.Measurement("cpu_temp").in_range(maximum=85).with_units(units.DEGREE_CELSIUS),
)
def test_cpu_board(test, serial):
    """Verify CPU board clock and temperature."""
    test.measurements.cpu_clock_mhz = float(serial.send_command("CLOCK?"))
    test.measurements.cpu_temp = float(serial.send_command("TEMP?"))


@htf.plug(adc=AnalogInputPlug)
@htf.measures(
    htf.Measurement("pwr_12v_rail").in_range(minimum=11.4, maximum=12.6).with_units(units.VOLT),
    htf.Measurement("pwr_efficiency_pct").in_range(minimum=85),
)
def test_power_board(test, adc):
    """Verify power board output rails and efficiency."""
    test.measurements.pwr_12v_rail = adc.read_voltage(channel=0)
    vin = adc.read_voltage(channel=4)
    vout = adc.read_voltage(channel=5)
    test.measurements.pwr_efficiency_pct = (vout / vin) * 100 if vin > 0 else 0


@htf.plug(serial=SerialCommandPlug, adc=AnalogInputPlug)
@htf.measures(
    htf.Measurement("sensor_offset_mv").in_range(minimum=-5, maximum=5),
    htf.Measurement("sensor_gain_error_pct").in_range(minimum=-1, maximum=1),
)
def test_sensor_board(test, serial, adc):
    """Verify sensor board calibration."""
    serial.send_command("SENSOR CAL_CHECK")
    test.measurements.sensor_offset_mv = float(serial.send_command("SENSOR OFFSET?"))
    test.measurements.sensor_gain_error_pct = float(serial.send_command("SENSOR GAIN_ERR?"))

Detecting Regressions in TofuPilot

TofuPilot tracks every measurement value across all runs. To detect firmware regressions:

Filter by firmware version. Open the procedure's Analytics tab and filter runs by the firmware_version measurement. Compare pass rates and measurement distributions between versions.
Check measurement trends. TofuPilot's trend charts show measurement values over time. A sudden shift after a firmware update is a clear regression signal.
Compare Cpk by version. If a measurement's Cpk drops after a firmware change, the process capability has degraded.

Manual Tracking vs TofuPilot

Aspect	Spreadsheet / Manual	TofuPilot
Firmware version tagging	Copy-paste into a column	Automatic per-run metadata
Waveform storage	Shared drive with naming conventions	Attached to the run, always findable
Multi-board traceability	Separate sheets or tabs	Sub-units linked to parent assembly
Regression detection	Manual chart inspection	Filter by firmware version, compare trends
Cross-station comparison	Merge files from different PCs	All stations upload to one workspace
Historical lookup	"Which folder was that in?"	Search by serial number, part number, or date
Audit trail	Hope nobody deleted a row	Immutable records with timestamps