Analyze Test Step Performance

A test that fails tells you something went wrong. Step-level analysis tells you exactly where. TofuPilot breaks down every run by phase so you can pinpoint which steps fail most, which take longest, and which measurements are closest to their limits.

Why Step-Level Analysis Matters

Overall pass/fail rates hide problems. A test with 95% FPY sounds fine until you discover one phase accounts for 80% of all failures. Without step-level data, you're guessing where to focus improvement efforts.

Step analysis answers three questions: Which phases have the lowest pass rate? Which measurements have the lowest Cpk? Which steps take the most time?

Structuring Tests for Good Step Data

Each OpenHTF phase becomes a step in TofuPilot. Name your phases clearly and keep each one focused on a single functional area. This makes the analysis meaningful.

multi_phase_test.py

import openhtf as htf
from openhtf.util import units
from tofupilot.openhtf import TofuPilot

@htf.measures(
    htf.Measurement("input_voltage")
    .with_units(units.VOLT)
    .in_range(minimum=11.5, maximum=12.5),
    htf.Measurement("input_current")
    .with_units(units.AMPERE)
    .in_range(maximum=2.0),
)
def power_input_check(test):
    """Verify power supply input within spec."""
    test.measurements.input_voltage = 12.03
    test.measurements.input_current = 1.45

@htf.measures(
    htf.Measurement("reg_3v3_output")
    .with_units(units.VOLT)
    .in_range(minimum=3.25, maximum=3.35),
    htf.Measurement("reg_5v_output")
    .with_units(units.VOLT)
    .in_range(minimum=4.90, maximum=5.10),
    htf.Measurement("reg_ripple")
    .in_range(maximum=50.0),
)
def voltage_regulator_test(test):
    """Test all onboard voltage regulators."""
    test.measurements.reg_3v3_output = 3.30
    test.measurements.reg_5v_output = 4.98
    test.measurements.reg_ripple = 22.4

@htf.measures(
    htf.Measurement("i2c_ack_received").with_allowed_values(True),
    htf.Measurement("spi_loopback_ok").with_allowed_values(True),
    htf.Measurement("uart_baud_error_pct")
    .in_range(maximum=2.0),
)
def communication_bus_test(test):
    """Verify I2C, SPI, and UART interfaces."""
    test.measurements.i2c_ack_received = True
    test.measurements.spi_loopback_ok = True
    test.measurements.uart_baud_error_pct = 0.8

@htf.measures(
    htf.Measurement("wifi_rssi")
    .in_range(minimum=-70, maximum=-20),
    htf.Measurement("bluetooth_pair_time")
    .in_range(maximum=5.0),
)
def wireless_connectivity_test(test):
    """Test WiFi and Bluetooth connections."""
    test.measurements.wifi_rssi = -45
    test.measurements.bluetooth_pair_time = 2.3

@htf.measures(
    htf.Measurement("flash_write_speed")
    .in_range(minimum=10.0),
    htf.Measurement("flash_read_speed")
    .in_range(minimum=20.0),
    htf.Measurement("eeprom_verify_ok").with_allowed_values(True),
)
def memory_test(test):
    """Verify flash and EEPROM read/write performance."""
    test.measurements.flash_write_speed = 14.2
    test.measurements.flash_read_speed = 28.7
    test.measurements.eeprom_verify_ok = True

def main():
    test = htf.Test(
        power_input_check,
        voltage_regulator_test,
        communication_bus_test,
        wireless_connectivity_test,
        memory_test,
    )
    with TofuPilot(test):
        test.execute(test_start=lambda: "BOARD-042")

if __name__ == "__main__":
    main()

This test has five distinct phases, each covering a different subsystem. TofuPilot tracks pass rates, measurement distributions, and timing for each one independently.

Finding Your Weakest Steps

In TofuPilot's step analysis view, sort phases by pass rate to find the weakest links. A phase with 92% pass rate in a five-phase test dominates your overall yield loss.

Look at the measurements within that phase. One measurement with a Cpk below 1.0 means your process variation is too wide for the spec. Either tighten the process or widen the spec if the product can tolerate it.

Identifying Slow Steps

Step timing data reveals bottlenecks. If your wireless connectivity test takes 8 seconds out of a 12-second total, that's where to focus cycle time reduction.

Common fixes for slow steps: reduce retry counts, parallelize independent measurements, or optimize instrument communication (batch queries instead of sequential reads).

Using Step Data to Prioritize

Combine pass rate, Cpk, and timing into a prioritized list. A step that fails often and runs slowly is the highest-impact target. A step with high Cpk and fast execution needs no attention.

TofuPilot's measurement analytics shows all of this per phase. Review it weekly to track whether your process improvements are actually moving the numbers.