Structure a Production Test Script

A test script that works on your bench and a test script that survives two years on a production line are different artefacts. The difference is almost entirely structural: what is separated from what. This guide covers the layout that holds up, and the specific couplings that cause trouble later.

The Four Things to Keep Apart

Most scripts that become unmaintainable have collapsed two or more of these into one file.

The Layout

One directory per procedure. Phases and plugs are ordinary Python modules, which means the tests/ directory can exercise them with pytest and no hardware.

Limits Belong in the Procedure, Not the Code

This is the separation that pays off most.

def test_power(dmm):    v = dmm.read_voltage()    if not 3.2 <= v <= 3.4:          # Limits buried in a comparison        raise AssertionError(f"Rail out of range: {v}")    return v

The value is checked and then discarded. The limits exist only inside an expression, so nothing can report them, compare against them, or tell you they changed in March.

main:  - name: Power Rails    python: phases.power_rails    measurements:      - name: Rail 3V3        unit: V        description: TP-100 rev B section 3.1        validators:          - operator: ">="            expected_value: 3.2          - operator: "<="            expected_value: 3.4

def power_rails(measurements, dmm):    measurements.rail_3v3 = dmm.read_voltage()

The phase does one thing: get a number and record it. Evaluation is the framework's job, and because the limits are data, a limit change is a reviewable diff rather than a code change.

Phases Should Be Boring

A good phase reads like a description of the step:

def firmware(measurements, dut):    measurements.firmware_version = dut.query_version()    measurements.bootloader_version = dut.query_bootloader()

Things that do not belong in a phase:

• Limit checks. Declare validators instead.
• Instrument setup. That is the plug's constructor.
• Retry loops. Use retry with a then: trigger in the procedure.
• Sleep-and-poll for an instrument. Put it in the plug's method.
• Deciding whether to run. Use enable_if or depends_on.

Each of those, left in a phase, becomes copy-pasted into the next twenty phases.

Plugs Own Lifecycle, Including Failure

import pyvisaclass PowerSupply:    RESOURCE = "TCPIP::192.168.1.101::INSTR"    def __init__(self):        rm = pyvisa.ResourceManager("@py")        self.instr = rm.open_resource(self.RESOURCE)        self.instr.timeout = 5000        self.instr.write("*RST")    def apply(self, channel: int, volts: float, limit: float):        self.instr.write(f":APPL CH{channel},{volts},{limit}")        self.instr.write(f":OUTP CH{channel},ON")    def close(self):        """Runs at teardown regardless of how the run ended."""        for ch in (1, 2, 3):            try:                self.instr.write(f":OUTP CH{ch},OFF")            except Exception:                pass        self.instr.close()

Putting power-down in close() rather than a final phase is the difference between a failed test that de-energises the board and one that leaves it powered on the fixture until someone notices.

Make Configuration External

The resource string above is hard-coded, which is fine until the second station has a different IP.

import osclass PowerSupply:    def __init__(self):        resource = os.environ.get("PSU_RESOURCE", "TCPIP::192.168.1.101::INSTR")        ...

Station-specific values are environment, not code. Product-specific values, such as limits, are the procedure. Keeping those two straight prevents the situation where a station has its own fork of the test.

Make Phases Testable Without Hardware

Because plugs are plain classes and phases are plain functions, both are testable in CI:

from phases.power_rails import power_railsclass FakeDmm:    def read_voltage(self) -> float:        return 3.31class Recorder:    def __init__(self):        self.values = {}    def __setattr__(self, name, value):        if name == "values":            super().__setattr__(name, value)        else:            self.values[name] = valuedef test_power_rails_records_voltage():    measurements = Recorder()    power_rails(measurements, FakeDmm())    assert measurements.values["rail_3v3"] == 3.31

This catches the class of bug that otherwise only appears on a station at 6 a.m.: a renamed measurement key, a typo'd attribute, a phase that silently records nothing.

Naming Is a Long-Term Commitment

Measurement names are the identity of your data across every run you will ever take. Renaming Rail 3V3 to 3V3 Rail splits the history in two and breaks every trend and capability calculation that spans the change.

Pick names early, make them specific (rail_3v3 not voltage), and leave them alone. If a name is genuinely wrong, changing it is a deliberate decision with a known cost, not a tidy-up.

Version the Procedure

name: PCBA Functional Testversion: 1.2.0description: FCT for PCBA-100, per TP-100 rev B

Every run records the version. Without it, a yield drop caused by a tightened limit is indistinguishable from a process problem, and that is a diagnosis that can take weeks.

Keep One Procedure for Attended and Unattended Runs

Maintaining a separate copy for CI guarantees the two drift. Use the same procedure and supply the operator answers externally:

tofupilot run ./procedure.yaml --json --ui-values ci/ui.json --ui-timeout 300

If You Already Run OpenHTF

The same separations apply, expressed with decorators. Keep plugs in their own module, phases in another, and the assembly in a thin entry point:

import openhtf as htffrom tofupilot.openhtf import uploadfrom phases import test_power_rails, test_firmwaredef main():    test = htf.Test(        test_power_rails,        test_firmware,        procedure_id="FCT-001",        part_number="PCBA-100",    )    test.add_output_callbacks(upload())    test.execute(test_start=lambda: input("Scan serial: "))if __name__ == "__main__":    main()

Install with pip install "tofupilot[openhtf]", or run it with tofupilot run.

Common Mistakes