# TofuPilot > Manufacturing test platform for hardware teams. Build and deploy hardware tests faster. ## Guides ### How to Compare Yield Between Lines URL: https://www.tofupilot.com/guides/how-to-compare-yield-between-production-lines-with-tofupilot Compare first-pass yield across production lines to identify underperforming equipment, operator training gaps, and process variation. Two production lines running the same product should produce the same yield. When they don't, the gap points to equipment issues, process variation, or operator training problems. TofuPilot lets you compare lines side by side without exporting data or building reports manually. ## Why Yield Comparison Matters A 2% yield difference between lines might look small. On 10,000 units per month, that's 200 extra failures, each costing rework time, replacement components, and delayed shipments. Catching the gap early and fixing the root cause pays for itself quickly. Common causes of line-to-line yield variation: | Cause | What to look for | |-------|-----------------| | Equipment calibration | One line's measurements consistently offset from the other | | Fixture wear | Increasing contact resistance on older test fixtures | | Operator technique | Manual steps done differently between shifts or lines | | Component lots | Different reels or batches feeding each line | | Environment | Temperature or humidity differences between areas | ## Tag Runs with Line Information Use `station_id` to encode which line produced each unit. TofuPilot uses this field to group and filter results. ```python filename="test_power_board.py" import openhtf as htf from openhtf.util import units from tofupilot.openhtf import TofuPilot @htf.measures( htf.Measurement("rail_3v3_voltage") .in_range(minimum=3.25, maximum=3.40) .with_units(units.VOLT), htf.Measurement("rail_5v0_voltage") .in_range(minimum=4.90, maximum=5.10) .with_units(units.VOLT), htf.Measurement("standby_current") .in_range(maximum=0.015) .with_units(units.AMPERE), ) def test_power_rails(test): test.measurements.rail_3v3_voltage = 3.31 test.measurements.rail_5v0_voltage = 5.02 test.measurements.standby_current = 0.0114 def main(): test = htf.Test(test_power_rails) with TofuPilot(test): test.execute(test_start=lambda: "PWR-2026-07823") if __name__ == "__main__": main() ``` Each station on Line A gets an ID like `LINE-A-FCT-01`, `LINE-A-FCT-02`. Line B uses `LINE-B-FCT-01`, `LINE-B-FCT-02`. This lets you filter at both the line level and the individual station level. ## Run Identical Tests on Every Line Yield comparison is only valid when every line runs the exact same test procedure with the same limits. Version-control your test scripts and deploy the same commit to all lines. ```python filename="test_comms_board.py" import openhtf as htf from openhtf.util import units from tofupilot.openhtf import TofuPilot @htf.measures( htf.Measurement("wifi_rssi") .in_range(minimum=-55.0), htf.Measurement("ble_rssi") .in_range(minimum=-65.0), htf.Measurement("antenna_impedance") .in_range(minimum=45.0, maximum=55.0) .with_units(units.OHM), ) def test_wireless(test): test.measurements.wifi_rssi = -42.3 test.measurements.ble_rssi = -51.7 test.measurements.antenna_impedance = 49.8 def main(): test = htf.Test(test_wireless) with TofuPilot(test): test.execute(test_start=lambda: "COMMS-2026-11045") if __name__ == "__main__": main() ``` If one line uses different firmware or a modified fixture, document that difference. Otherwise your yield comparison is measuring two different things. ## Compare Lines in TofuPilot With runs uploading from both lines, TofuPilot's filtering and analytics give you direct comparison: - **FPY by station** shows yield for each station grouped by line prefix. If Line B consistently trails Line A, the problem is systemic to that line. - **Measurement histograms** reveal whether one line's values are shifted or have wider spread. A shifted mean suggests calibration offset. Wider spread suggests fixture or process inconsistency. - **Failure Pareto by line** shows which specific tests fail more often on each line. If Line B fails `antenna_impedance` three times more than Line A, start investigating that fixture. - **Trend charts** show whether the gap is constant, growing, or appeared suddenly after a change. ## Investigate and Act on Differences When you find a yield gap, narrow down the cause systematically: 1. **Check measurement distributions.** If one line's values are offset, it's likely calibration or equipment. If they're wider, it's process variation. 2. **Check by time of day.** Yield drops on night shifts point to operator training or environmental changes. 3. **Check by component lot.** If you track lot numbers, filter by lot to see if specific batches drive the difference. 4. **Check individual stations.** Sometimes the "line" problem is actually one bad station dragging down the average. Fix the root cause, then watch TofuPilot's trend data to confirm the gap closes. The measurement-level detail in each run gives you the evidence to verify the fix worked. ### Environmental Testing with TofuPilot URL: https://www.tofupilot.com/guides/environmental-testing-with-tofupilot Learn how to log thermal cycling, humidity, HALT, and environmental screening test results in TofuPilot for reliability tracking. # Environmental Testing with TofuPilot Products fail in the field because of temperature, humidity, vibration, and altitude. Environmental testing simulates these conditions in the lab. TofuPilot tracks every environmental test result so you can build a reliability picture across your entire product lifecycle. ## Types of Environmental Tests | Test | What it simulates | Standards | |------|------------------|-----------| | Thermal cycling | Temperature extremes during shipping and operation | IEC 60068-2-14, MIL-STD-810 | | Thermal shock | Rapid temperature transitions | IEC 60068-2-14 | | Humidity testing | Moisture exposure, condensation | IEC 60068-2-78 | | HALT (Highly Accelerated Life Test) | Find design weaknesses fast | No formal standard | | HASS (Highly Accelerated Stress Screen) | Production screening | No formal standard | | Vibration | Mechanical stress during transport and operation | IEC 60068-2-6, IEC 60068-2-64 | | Altitude | Low pressure environments | IEC 60068-2-13 | | Salt spray | Corrosion resistance | IEC 60068-2-11 | ## Logging Thermal Cycling Results A thermal cycling test runs a product through temperature extremes for hundreds of cycles. After each cycle (or batch of cycles), perform a functional check and log results. ```python filename="thermal_cycling_test.py" from tofupilot import TofuPilotClient client = TofuPilotClient() def log_thermal_cycle_check(serial, cycle_number, temp_profile, functional_results): """Log post-cycle functional check results.""" measurements = [ {"name": "cycle_number", "value": cycle_number, "unit": "cycles"}, {"name": "temp_high_c", "value": temp_profile["high"], "unit": "°C"}, {"name": "temp_low_c", "value": temp_profile["low"], "unit": "°C"}, {"name": "dwell_time_min", "value": temp_profile["dwell"], "unit": "min"}, ] # Add functional measurements taken after thermal exposure for name, result in functional_results.items(): measurements.append({ "name": name, "value": result["value"], "unit": result["unit"], "limit_low": result.get("limit_low"), "limit_high": result.get("limit_high"), }) all_pass = all( r.get("limit_low", float("-inf")) <= r["value"] <= r.get("limit_high", float("inf")) for r in functional_results.values() ) client.create_run( procedure_id="THERMAL-CYCLING-500", unit_under_test={"serial_number": serial}, run_passed=all_pass, steps=[{ "name": f"Post-Cycle {cycle_number} Functional Check", "step_type": "measurement", "status": all_pass, "measurements": measurements, }], ) # Example: check after every 50 cycles for cycle in range(50, 501, 50): functional = { "vcc_3v3": {"value": measure_voltage(), "unit": "V", "limit_low": 3.25, "limit_high": 3.35}, "current_ma": {"value": measure_current() * 1000, "unit": "mA", "limit_low": 30, "limit_high": 60}, "comm_check": {"value": 1 if uart_ok() else 0, "unit": "bool", "limit_low": 1}, } log_thermal_cycle_check( serial="EVT-UNIT-003", cycle_number=cycle, temp_profile={"high": 85, "low": -40, "dwell": 15}, functional_results=functional, ) ``` ## Tracking Degradation Over Cycles The key question in environmental testing: "At what point does the product start degrading?" TofuPilot's measurement trending shows how each parameter changes across thermal cycles: - **Cycle 50**: vcc_3v3 = 3.31V (nominal) - **Cycle 200**: vcc_3v3 = 3.30V (still nominal) - **Cycle 350**: vcc_3v3 = 3.28V (drifting) - **Cycle 450**: vcc_3v3 = 3.26V (approaching limit) - **Cycle 500**: vcc_3v3 = 3.24V (below limit, FAIL) This degradation curve tells you the product survives 400+ cycles before the 3.3V rail drifts out of spec. If your spec requires 500 cycles, you need a design change. ## HALT Testing HALT finds design weaknesses by pushing products beyond their specifications. There are no pass/fail limits. The goal is to find the breaking point. ```python filename="halt_test.py" # HALT step stress results halt_steps = [ {"stress": "Cold Step", "temp_c": -40, "functional": True}, {"stress": "Cold Step", "temp_c": -50, "functional": True}, {"stress": "Cold Step", "temp_c": -60, "functional": False}, # Lower operating limit found {"stress": "Hot Step", "temp_c": 85, "functional": True}, {"stress": "Hot Step", "temp_c": 95, "functional": True}, {"stress": "Hot Step", "temp_c": 105, "functional": False}, # Upper operating limit found {"stress": "Vibration", "grms": 10, "functional": True}, {"stress": "Vibration", "grms": 20, "functional": True}, {"stress": "Vibration", "grms": 30, "functional": False}, # Vibration limit found ] steps = [] for h in halt_steps: steps.append({ "name": f"{h['stress']} - {'temp_c' in h and f\"{h['temp_c']}°C\" or f\"{h['grms']}Grms\"}", "step_type": "measurement", "status": h["functional"], "measurements": [{ "name": "functional_after_stress", "value": 1 if h["functional"] else 0, "unit": "pass/fail", }], }) client.create_run( procedure_id="HALT-STEP-STRESS", unit_under_test={"serial_number": "EVT-UNIT-001"}, run_passed=True, # HALT always "passes" - it's about finding limits steps=steps, ) ``` ## Environmental Test Matrix Track which environmental tests have been completed across your product variants. | Product variant | Thermal cycle | Humidity | Vibration | HALT | ESD | |----------------|---------------|----------|-----------|------|-----| | Rev A | 500 cycles ✓ | 85/85 ✓ | 10G ✓ | Done ✓ | 8kV ✓ | | Rev B | 250/500 | Pending | Pending | Done ✓ | 8kV ✓ | | Rev C (new layout) | Not started | Not started | Not started | Pending | Not started | TofuPilot tracks completion automatically. If `THERMAL-CYCLING-500` has a passing run for "Rev A," it shows as complete. No spreadsheet maintenance needed. ## Production Screening (HASS/ESS) Environmental stress screening in production catches infant mortality failures. Run every unit through a short thermal cycle and vibration profile, then functional test. ```python filename="production_screen.py" # Quick production screen: 10 thermal cycles + vibration client.create_run( procedure_id="ESS-PRODUCTION-SCREEN", unit_under_test={"serial_number": "PROD-UNIT-4521"}, run_passed=True, steps=[ { "name": "Post-Thermal Functional", "step_type": "measurement", "status": True, "measurements": [ {"name": "vcc_3v3", "value": 3.31, "unit": "V", "limit_low": 3.25, "limit_high": 3.35}, {"name": "current_ma", "value": 44, "unit": "mA", "limit_low": 30, "limit_high": 60}, ], }, { "name": "Post-Vibration Functional", "step_type": "measurement", "status": True, "measurements": [ {"name": "vcc_3v3", "value": 3.30, "unit": "V", "limit_low": 3.25, "limit_high": 3.35}, {"name": "current_ma", "value": 45, "unit": "mA", "limit_low": 30, "limit_high": 60}, ], }, ], ) ``` Track ESS yield separately from functional test yield. If ESS catches 0.5% of units that passed functional test, it's paying for itself by preventing field failures. ### Getting Started with PyVISA URL: https://www.tofupilot.com/guides/getting-started-with-pyvisa-control-test-instruments-from-python Learn how to connect to and control test instruments (DMMs, oscilloscopes, power supplies) from Python using PyVISA, with TofuPilot measurement logging. PyVISA lets you control test instruments (multimeters, oscilloscopes, power supplies, signal generators) from Python using the same VISA protocol that LabVIEW and TestStand use. If your instrument has a USB, GPIB, Ethernet, or serial interface, you can talk to it with PyVISA. This guide covers setup, connection, SCPI commands, and integrating instrument measurements into a TofuPilot test. ## Prerequisites - Python 3.8+ - A test instrument with USB-TMC, GPIB, Ethernet (LXI), or serial interface - Optional: NI-VISA runtime (for GPIB and some USB instruments) ## Installation ```python filename="pyvisa_setup/install.sh" pip install pyvisa pyvisa-py ``` Two backends are available: | Backend | Install | When to Use | |---------|---------|-------------| | **pyvisa-py** | `pip install pyvisa-py` | Pure Python. Works on Linux/macOS/Windows. Supports USB-TMC, Ethernet (TCP/IP), serial. No NI software needed. | | **NI-VISA** | Install from ni.com | Required for GPIB. Also supports USB-TMC and Ethernet. Windows and Linux only. | Start with pyvisa-py. Switch to NI-VISA only if you need GPIB or encounter compatibility issues. ## Finding Your Instruments Every VISA instrument has a resource string that identifies it. PyVISA can discover connected instruments automatically. ```python filename="pyvisa_setup/discover.py" import pyvisa rm = pyvisa.ResourceManager("@py") # Use pyvisa-py backend # rm = pyvisa.ResourceManager() # Use NI-VISA backend # List all connected instruments resources = rm.list_resources() print(f"Found {len(resources)} instrument(s):") for r in resources: print(f" {r}") ``` Common resource string formats: | Interface | Resource String | Example | |-----------|----------------|---------| | USB-TMC | `USB0::VID::PID::SERIAL::INSTR` | `USB0::0x2A8D::0x1301::MY59001234::INSTR` | | Ethernet (LXI) | `TCPIP::IP::INSTR` | `TCPIP::192.168.1.100::INSTR` | | GPIB | `GPIB0::ADDR::INSTR` | `GPIB0::22::INSTR` | | Serial | `ASRL/dev/ttyUSB0::INSTR` | `ASRL/dev/ttyUSB0::INSTR` | ## Connecting to an Instrument Once you have the resource string, open a connection and verify identity with the standard `*IDN?` query. ```python filename="pyvisa_setup/connect.py" import pyvisa rm = pyvisa.ResourceManager("@py") dmm = rm.open_resource("TCPIP::192.168.1.100::INSTR") # Set timeout (milliseconds) dmm.timeout = 5000 # Query instrument identity idn = dmm.query("*IDN?") print(f"Connected to: {idn.strip()}") # Output: Keysight Technologies,34461A,MY59001234,A.03.01 ``` The `query()` method sends a command and reads the response. For commands that don't return data, use `write()`. ## SCPI Command Basics SCPI (Standard Commands for Programmable Instruments) is the language most modern instruments speak. Commands follow a tree structure. ### Reading a DC Voltage ```python filename="pyvisa_setup/measure_voltage.py" import pyvisa rm = pyvisa.ResourceManager("@py") dmm = rm.open_resource("TCPIP::192.168.1.100::INSTR") dmm.timeout = 5000 # Configure for DC voltage measurement, auto-range dmm.write(":CONF:VOLT:DC AUTO") # Trigger a measurement and read the result dmm.write(":INIT") voltage = float(dmm.query(":FETCH?")) print(f"Voltage: {voltage:.4f} V") dmm.close() rm.close() ``` ### Common SCPI Commands | Command | Purpose | Example | |---------|---------|---------| | `*IDN?` | Identify instrument | Returns manufacturer, model, serial, firmware | | `*RST` | Reset to factory defaults | Good practice at test start | | `*CLS` | Clear error queue | Clear any previous errors | | `*OPC?` | Operation complete query | Returns "1" when last command finishes | | `:CONF:VOLT:DC` | Configure DC voltage | `:CONF:VOLT:DC 10` for 10V range | | `:CONF:CURR:DC` | Configure DC current | `:CONF:CURR:DC AUTO` for auto-range | | `:CONF:RES` | Configure resistance | `:CONF:RES AUTO` | | `:MEAS:VOLT:DC?` | Measure DC voltage (configure + trigger + read) | Returns float | | `:INIT` | Trigger measurement | Use with `:FETCH?` for separate trigger/read | | `:FETCH?` | Read last measurement | Returns float | | `:SYST:ERR?` | Read error queue | Returns error code and message | The `:MEAS:` shorthand configures, triggers, and reads in one command. Use `:CONF:` + `:INIT` + `:FETCH?` when you need more control over timing. ## Controlling a Power Supply Power supplies use similar SCPI commands but add output control. ```python filename="pyvisa_setup/power_supply.py" import pyvisa import time rm = pyvisa.ResourceManager("@py") psu = rm.open_resource("TCPIP::192.168.1.101::INSTR") psu.timeout = 5000 # Reset to known state psu.write("*RST") psu.write("*CLS") # Configure channel 1: 5V, 500mA current limit psu.write(":INST:SEL CH1") psu.write(":VOLT 5.0") psu.write(":CURR 0.5") # Enable output psu.write(":OUTP ON") time.sleep(0.5) # Wait for output to stabilize # Read actual voltage and current actual_voltage = float(psu.query(":MEAS:VOLT?")) actual_current = float(psu.query(":MEAS:CURR?")) print(f"Output: {actual_voltage:.3f}V, {actual_current:.4f}A") # Disable output psu.write(":OUTP OFF") psu.close() rm.close() ``` ## Error Handling Instruments communicate errors through the SCPI error queue. Always check for errors after a sequence of commands. ```python filename="pyvisa_setup/error_handling.py" import pyvisa def check_instrument_errors(instr): """Read and report all errors from the instrument error queue.""" errors = [] while True: err = instr.query(":SYST:ERR?").strip() code = int(err.split(",")[0]) if code == 0: break errors.append(err) return errors rm = pyvisa.ResourceManager("@py") dmm = rm.open_resource("TCPIP::192.168.1.100::INSTR") dmm.timeout = 5000 # Send commands dmm.write("*RST") dmm.write(":CONF:VOLT:DC AUTO") dmm.write(":INIT") voltage = float(dmm.query(":FETCH?")) # Check for errors errors = check_instrument_errors(dmm) if errors: print(f"Instrument errors: {errors}") else: print(f"Voltage: {voltage:.4f} V (no errors)") dmm.close() ``` ## Integrating with OpenHTF and TofuPilot In a production test, instrument control lives inside an OpenHTF Plug. The plug manages the connection lifecycle, and measurements flow through OpenHTF into TofuPilot automatically. ```python filename="pyvisa_setup/production_test.py" import pyvisa import openhtf as htf from openhtf.plugs import BasePlug from openhtf.util import units from tofupilot.openhtf import TofuPilot class MultimeterPlug(BasePlug): """PyVISA multimeter plug with automatic lifecycle management.""" RESOURCE = "TCPIP::192.168.1.100::INSTR" def setUp(self): rm = pyvisa.ResourceManager("@py") self.instr = rm.open_resource(self.RESOURCE) self.instr.timeout = 5000 self.instr.write("*RST") self.instr.write("*CLS") self.logger.info(f"Connected: {self.instr.query('*IDN?').strip()}") def measure_dc_voltage(self, range_v: str = "AUTO") -> float: """Take a single DC voltage measurement.""" self.instr.write(f":CONF:VOLT:DC {range_v}") return float(self.instr.query(":MEAS:VOLT:DC?")) def measure_dc_current(self, range_a: str = "AUTO") -> float: """Take a single DC current measurement.""" self.instr.write(f":CONF:CURR:DC {range_a}") return float(self.instr.query(":MEAS:CURR:DC?")) def measure_resistance(self, range_ohm: str = "AUTO") -> float: """Take a single resistance measurement.""" self.instr.write(f":CONF:RES {range_ohm}") return float(self.instr.query(":MEAS:RES?")) def tearDown(self): self.instr.close() self.logger.info("Multimeter disconnected") @htf.measures( htf.Measurement("supply_3v3") .in_range(3.2, 3.4) .with_units(units.VOLT) .doc("3.3V rail voltage"), htf.Measurement("supply_5v0") .in_range(4.8, 5.2) .with_units(units.VOLT) .doc("5.0V rail voltage"), htf.Measurement("idle_current") .in_range(0.01, 0.15) .with_units(units.AMPERE) .doc("Board idle current draw"), ) @htf.plug(dmm=MultimeterPlug) def test_power_rails(test, dmm): """Measure all power rails and idle current.""" test.measurements.supply_3v3 = dmm.measure_dc_voltage() test.measurements.supply_5v0 = dmm.measure_dc_voltage() test.measurements.idle_current = dmm.measure_dc_current() @htf.measures( htf.Measurement("pullup_resistance") .in_range(4500, 5500) .with_units(units.OHM) .doc("I2C pullup resistance (expect 4.7k)"), ) @htf.plug(dmm=MultimeterPlug) def test_pullup_resistors(test, dmm): """Verify I2C pullup resistor values.""" test.measurements.pullup_resistance = dmm.measure_resistance() def main(): test = htf.Test( test_power_rails, test_pullup_resistors, procedure_id="FCT-001", part_number="PCBA-100", ) with TofuPilot(test): test.execute(test_start=lambda: input("Scan serial number: ")) if __name__ == "__main__": main() ``` Every measurement (voltage, current, resistance) flows into TofuPilot with its name, value, limits, and units. You get FPY tracking, control charts, and Cpk analysis on each measurement without extra code. ## Troubleshooting | Problem | Cause | Fix | |---------|-------|-----| | `VisaIOError: VI_ERROR_RSRC_NFOUND` | Instrument not found | Check cable, verify IP/USB, run `rm.list_resources()` | | `VisaIOError: VI_ERROR_TMO` | Timeout | Increase `instr.timeout`, check if instrument is busy | | `VisaIOError: VI_ERROR_CONN_LOST` | Connection dropped | Check network, retry connection | | Empty response from `query()` | Instrument didn't respond | Add `time.sleep()` after write, check command syntax | | USB instrument not detected | Missing driver | Install NI-VISA runtime or check USB-TMC kernel module (Linux) | | `ValueError: could not convert string to float` | Unexpected response format | Print raw response, check for error messages mixed in | ### Pareto Analysis for Test Failures URL: https://www.tofupilot.com/guides/how-to-use-pareto-analysis-for-test-failures-with-tofupilot Learn how the 80/20 Pareto principle applies to test failures, how to write tests that produce actionable failure data, and how to use TofuPilot's Pareto chart. The Pareto principle says 80% of your test failures come from 20% of your failure modes. If you fix the right two or three issues, most of your yield problems disappear. But you need structured failure data to find those issues, and that starts with how you write your tests. ## What Pareto Analysis Tells You A Pareto chart ranks failure modes by frequency, highest to lowest, with a cumulative line showing how much of total failures each mode accounts for. It answers one question: where should you spend your time? Without a Pareto chart, teams chase whatever failure they saw most recently. With one, you can see that "clock frequency out of range" causes 40% of all failures while "communication timeout" causes 3%. Fixing the clock issue first gives you 13x more impact. ## Writing Tests with Clear Failure Modes Pareto analysis only works if your test phases map to distinct, diagnosable failure modes. A single phase that checks 15 things produces one failure label when any of them breaks. You can't tell which sub-check caused it. Split your test into phases that each cover one functional area. ```python filename="multi_phase_test.py" import openhtf as htf from openhtf.util import units from tofupilot.openhtf import TofuPilot @htf.measures( htf.Measurement("input_voltage") .with_units(units.VOLT) .in_range(minimum=4.75, maximum=5.25), htf.Measurement("inrush_current") .in_range(maximum=500), ) def test_power_input(test): test.measurements.input_voltage = 5.02 test.measurements.inrush_current = 320 @htf.measures( htf.Measurement("rail_3v3_voltage") .with_units(units.VOLT) .in_range(minimum=3.135, maximum=3.465), htf.Measurement("rail_3v3_ripple") .in_range(maximum=50), ) def test_3v3_regulator(test): test.measurements.rail_3v3_voltage = 3.28 test.measurements.rail_3v3_ripple = 22 @htf.measures( htf.Measurement("clock_frequency") .in_range(minimum=7.99, maximum=8.01), htf.Measurement("clock_jitter") .in_range(maximum=100), ) def test_clock(test): test.measurements.clock_frequency = 8.003 test.measurements.clock_jitter = 45 @htf.measures( htf.Measurement("spi_loopback_pass"), htf.Measurement("spi_throughput") .in_range(minimum=900), ) def test_spi_interface(test): test.measurements.spi_loopback_pass = True test.measurements.spi_throughput = 980 @htf.measures( htf.Measurement("adc_offset_error") .in_range(minimum=-2, maximum=2), htf.Measurement("adc_gain_error_pct") .in_range(minimum=-0.5, maximum=0.5), htf.Measurement("adc_snr") .in_range(minimum=60), ) def test_adc_accuracy(test): test.measurements.adc_offset_error = 0.8 test.measurements.adc_gain_error_pct = -0.12 test.measurements.adc_snr = 67.3 def main(): test = htf.Test( test_power_input, test_3v3_regulator, test_clock, test_spi_interface, test_adc_accuracy, ) with TofuPilot(test): test.execute(test_start=lambda: "SN-2026-01580") if __name__ == "__main__": main() ``` Each phase covers one functional block: power input, voltage regulation, clock, SPI communication, and ADC accuracy. When a unit fails, the failure is tagged to a specific phase and measurement. TofuPilot uses this structure to build the Pareto chart. ## Reading TofuPilot's Failure Pareto Chart TofuPilot's analytics dashboard includes a failure Pareto chart that updates in real time as test data flows in. Here's how to use it. **Select your scope.** Filter by product, station, or time range. A Pareto chart for all products combined isn't useful because different products have different failure profiles. **Read the bars.** Each bar represents a failure mode (a test phase or specific measurement that failed). The tallest bar on the left is your biggest problem. **Follow the cumulative line.** The line shows what percentage of total failures you've covered as you move left to right. If the first two bars reach 70% on the cumulative line, fixing those two issues eliminates 70% of your failures. **Compare over time.** After fixing the top failure mode, check the Pareto chart again. The distribution will shift. What was previously the second-biggest issue might now be the top one, or a previously hidden issue might surface. ## Turning Pareto Data into Action A Pareto chart gives you priorities. Turning those priorities into fixes requires a systematic approach. ### For the Top Failure Mode 1. Pull up the measurement histogram in TofuPilot for the failing measurement. Check whether failures cluster at one limit or spread across the range. 2. Check the Cpk. If Cpk is below 1.0, your process isn't capable of meeting the test limits consistently. You either need to improve the process or review whether the limits are too tight. 3. Look at the unit history for several failed serial numbers. Do they fail once and pass on retest (fixture issue), or do they fail consistently (real defect)? 4. Check for station correlation. If failures concentrate on one station, the problem is likely the fixture or test environment, not the product. ### For Recurring Low-Level Failures Some failure modes sit at 2-5% individually but collectively account for 20-30% of failures. These are often fixture wear, environmental drift, or operator handling issues. Address them as a group by improving fixture maintenance schedules and adding fixture validation phases to your tests. ## What Good Looks Like A healthy Pareto chart has no single failure mode dominating above 15-20%. The bars taper gradually from left to right, indicating no single systemic issue. If your chart shows one bar towering over the rest, you have a known problem that hasn't been addressed. If the chart is nearly flat with many small bars, your failures are distributed and likely random, which means your process is well-controlled and further improvement requires tightening tolerances or upgrading components. Review the Pareto chart weekly during DVT and PVT. In production, a monthly review catches new trends before they become costly. ### What Is Automated Test Equipment (ATE) URL: https://www.tofupilot.com/guides/what-is-ate-with-tofupilot Automated test equipment (ATE) runs production tests without manual intervention. Learn what ATE involves, how to build Python-based systems, and track results. # What Is ATE with TofuPilot Automated test equipment (ATE) is a system that tests a device without manual intervention. It applies stimuli, measures responses, and makes pass/fail decisions based on predefined limits. This guide covers what ATE involves, how modern Python-based ATE compares to traditional systems, and how to log ATE results with TofuPilot. ## What ATE Includes An ATE system has four layers: | Layer | Purpose | Examples | |-------|---------|---------| | Test executive | Sequences test steps, manages flow | OpenHTF, NI TestStand, custom scripts | | Instruments | Apply stimuli and measure responses | DMM, oscilloscope, power supply, signal generator | | Fixture | Connects instruments to the DUT | Bed-of-nails, pogo pins, cable harness | | Software | Controls instruments, records data | Python + PyVISA, LabVIEW, C# | The device under test (DUT) or unit under test (UUT) sits in the fixture. The test executive runs the sequence. Instruments measure. Software records. ## Traditional ATE vs Python-Based ATE | Aspect | Traditional (NI/Keysight) | Python-Based | |--------|--------------------------|-------------| | Test executive | NI TestStand ($3-5K/seat) | OpenHTF (free, open source) | | Instrument control | LabVIEW, proprietary drivers | PyVISA, SCPI, open drivers | | Data storage | Local database, proprietary format | TofuPilot (cloud or self-hosted) | | Version control | Difficult with binary files | Git-native (Python scripts) | | Platform | Windows only | Windows, Linux, macOS | | Deployment | Manual install per station | pip install, Docker, CI/CD | | Cost per station | $5-20K in software licenses | $0 in software licenses | Python-based ATE uses the same instruments and fixtures. The difference is the software stack. You replace proprietary test executives and data systems with open-source tools and TofuPilot. ## Prerequisites - Python 3.10+ - OpenHTF installed (`pip install openhtf`) - TofuPilot Python SDK installed (`pip install tofupilot`) ## Step 1: Define the Test Sequence Each test step becomes an OpenHTF phase. The test executive runs them in order, collects measurements, and determines pass/fail. ```python filename="ate_test.py" import openhtf as htf from openhtf.util import units @htf.measures( htf.Measurement("supply_current_mA") .in_range(minimum=90, maximum=110) .with_units(units.MILLIAMPERE), ) def phase_power_up(test): """Apply power and measure supply current.""" test.measurements.supply_current_mA = 101.3 @htf.measures( htf.Measurement("output_frequency_Hz") .in_range(minimum=999000, maximum=1001000) .with_units(units.HERTZ), ) def phase_frequency_check(test): """Measure output frequency against specification.""" test.measurements.output_frequency_Hz = 1000250 @htf.measures( htf.Measurement("self_test_result").equals("PASS"), ) def phase_self_test(test): """Command the DUT to run its built-in self-test.""" test.measurements.self_test_result = "PASS" ``` ## Step 2: Connect to TofuPilot TofuPilot replaces the proprietary database that traditional ATE systems use. Every test run uploads automatically with measurements, limits, and pass/fail status. ```python filename="ate_test.py" from tofupilot.openhtf import TofuPilot test = htf.Test( phase_power_up, phase_frequency_check, phase_self_test, ) with TofuPilot(test): test.execute(test_start=lambda: input("Scan DUT serial: ")) ``` ## Step 3: Track ATE Performance TofuPilot tracks ATE results automatically. Open the Analytics tab to see: - **First pass yield** per test procedure and station - **Measurement distributions** with limit overlays - **Failure Pareto** showing which test steps fail most - **Station throughput** (units per hour) - **Station comparison** to catch fixture degradation or instrument drift This data replaces the custom reports that traditional ATE software generates. It's available across all stations in real time. ## ATE Architecture Patterns | Pattern | Stations | Best For | |---------|----------|----------| | Single station, single DUT | 1 | Prototyping, low volume | | Single station, multi-DUT | 1 | Parallel testing, higher throughput | | Multi-station, shared fixtures | 2-10 | Medium volume production | | Multi-station, line integration | 10-100 | High volume, MES integration | TofuPilot supports all patterns. Each station runs its own test script and uploads results independently. The dashboard aggregates data across stations, lines, and factories. ### OpenHTF vs pytest for Hardware Testing URL: https://www.tofupilot.com/guides/openhtf-vs-pytest-for-hardware-testing-which-should-you-choose A structured comparison of OpenHTF and pytest for manufacturing test automation, with code examples, tradeoffs, and guidance on when to use each framework. OpenHTF and pytest are the two most common Python frameworks for hardware test automation. They solve different problems. OpenHTF was built by Google for manufacturing test. pytest was built for software testing and adapted by hardware teams. This guide compares them side by side with real code so you can pick the right one. ## Feature Comparison | Feature | OpenHTF | pytest | |---------|---------|--------| | **Designed for** | Manufacturing/production test | Software testing (adapted for hardware) | | **Test structure** | Phases (ordered sequence) | Functions (unordered by default) | | **Measurements** | Built-in: name, value, limits, units | Manual: assert statements or custom fixtures | | **Serial number input** | Built-in (operator prompt) | Manual implementation | | **Operator UI** | Built-in web UI | None (third-party or custom) | | **Instrument lifecycle** | Plugs (auto setup/teardown) | Fixtures (similar, more flexible) | | **Attachments** | Built-in (files, images, logs) | Manual (save to disk or custom plugin) | | **Test report** | Structured protobuf output | JUnit XML, custom plugins | | **Parallel DUT testing** | Limited (single test executor) | Native (pytest-xdist) | | **Community size** | Small (~640 GitHub stars) | Massive (11K+ stars, huge ecosystem) | | **Plugin ecosystem** | Minimal | Thousands of plugins | | **Learning curve** | Steeper (less documentation) | Gentler (extensive documentation) | | **TofuPilot integration** | Native (1 line of code) | Via Python SDK | ## The Same Test in Both Frameworks Here's a functional test that verifies a PCBA's power rail and communication interface, written in both frameworks. ### OpenHTF Version ```python filename="comparison/openhtf_test.py" import openhtf as htf from openhtf.plugs import BasePlug from openhtf.util import units from tofupilot.openhtf import TofuPilot class DutPlug(BasePlug): """Manage DUT connection lifecycle.""" def setUp(self): self.connected = True # Replace with real connection def read_voltage(self) -> float: return 3.31 # Replace with instrument read def query_firmware(self) -> str: return "2.1.0" # Replace with DUT query def tearDown(self): self.connected = False @htf.measures( htf.Measurement("rail_3v3") .in_range(3.2, 3.4) .with_units(units.VOLT) .doc("3.3V supply rail voltage"), ) @htf.PhaseOptions(timeout_s=10) @htf.plug(dut=DutPlug) def test_power(test, dut): test.measurements.rail_3v3 = dut.read_voltage() @htf.measures( htf.Measurement("firmware_version") .equals("2.1.0") .doc("Expected firmware version string"), ) @htf.PhaseOptions(timeout_s=5) @htf.plug(dut=DutPlug) def test_firmware(test, dut): test.measurements.firmware_version = dut.query_firmware() def main(): test = htf.Test( test_power, test_firmware, procedure_id="FCT-001", part_number="PCBA-100", ) with TofuPilot(test): test.execute(test_start=lambda: input("Scan serial number: ")) if __name__ == "__main__": main() ``` ### pytest Version ```python filename="comparison/pytest_test.py" import pytest from tofupilot import TofuPilotClient @pytest.fixture(scope="session") def tofupilot(): """Initialize TofuPilot client for the test session.""" return TofuPilotClient() @pytest.fixture(scope="session") def serial_number(): """Prompt operator for serial number before tests run.""" return input("Scan serial number: ") @pytest.fixture def dut(): """Manage DUT connection. Equivalent to an OpenHTF Plug.""" # Setup connection = {"connected": True} # Replace with real connection yield connection # Teardown connection["connected"] = False def read_voltage() -> float: """Read voltage from instrument.""" return 3.31 # Replace with real read def query_firmware() -> str: """Query firmware version from DUT.""" return "2.1.0" # Replace with real query def test_power_rail(dut): """Verify 3.3V supply rail is within spec.""" voltage = read_voltage() assert 3.2 <= voltage <= 3.4, f"3.3V rail out of range: {voltage}V" def test_firmware_version(dut): """Verify firmware version matches expected.""" version = query_firmware() assert version == "2.1.0", f"Unexpected firmware: {version}" ``` ### Key Differences in the Code | Aspect | OpenHTF | pytest | |--------|---------|--------| | **Measurements** | Declarative with `@htf.measures`. Name, limits, units defined upfront. | Implicit via `assert`. No structured metadata. | | **Serial number** | Built into `test.execute()`. | Custom fixture or input call. | | **DUT lifecycle** | Plug class with `setUp`/`tearDown`. Injected via `@htf.plug()` decorator. | Fixture with `yield`. Manually passed. | | **Limits** | `.in_range(3.2, 3.4)` stored as structured data. | `assert 3.2 <= v <= 3.4` is code, not data. | | **Output** | Structured protobuf with all metadata. | Pass/fail only (unless you add custom reporting). | | **Timeouts** | `@htf.PhaseOptions(timeout_s=10)` | `@pytest.mark.timeout(10)` (requires plugin) | ## When to Use OpenHTF OpenHTF is the better choice when: - **You're running production tests.** The built-in measurement model (name, value, limits, units) maps directly to how manufacturing test data needs to be stored and analyzed. You don't need to build this yourself. - **You need operator interaction.** OpenHTF has a built-in web UI for operators to scan serial numbers, see test progress, and view results. With pytest, you'd build this from scratch. - **You want structured test data.** Every measurement in OpenHTF carries its name, value, limits, units, and pass/fail status. This feeds directly into TofuPilot for FPY, Cpk, and control chart analysis. With pytest, you get pass/fail per test function, nothing more. - **Your team already uses OpenHTF.** If you have existing OpenHTF tests, stay with it. The migration cost isn't worth the switch. ## When to Use pytest pytest is the better choice when: - **You're doing R&D or validation testing.** pytest's flexibility shines when test requirements change frequently. No strict phase ordering, easy to skip or select tests, rich assertion messages. - **You need parallel DUT testing.** pytest-xdist runs tests in parallel natively. OpenHTF's executor is single-threaded. - **Your team knows pytest.** Most Python developers already know pytest. OpenHTF has a learning curve and sparse documentation. - **You want a plugin ecosystem.** pytest-timeout, pytest-repeat, pytest-html, pytest-cov. Thousands of plugins for every need. OpenHTF has almost none. - **You're testing firmware/software on hardware.** If the "test" is really a software test that happens to run on hardware (flashing firmware, running integration tests on a dev board), pytest is the natural fit. ## Using Both Together Some teams use both. pytest for firmware validation and CI, OpenHTF for production functional test. This works well when the test requirements are genuinely different: | Test Stage | Framework | Why | |-----------|-----------|-----| | Firmware CI | pytest | Runs in CI pipeline, parallel execution, software-style testing | | EVT/DVT validation | pytest | Flexible, exploratory, requirements change frequently | | PVT/Production FCT | OpenHTF | Structured measurements, operator UI, production data logging | | Incoming inspection | OpenHTF | Repeatable, operator-driven, needs traceability | Both frameworks work with TofuPilot. OpenHTF has native integration (one line). pytest works through the Python SDK (a few more lines, same data). ## TofuPilot Integration Comparison ### OpenHTF: One line ```python filename="comparison/tofupilot_openhtf.py" from tofupilot.openhtf import TofuPilot # Wrap your test execution with TofuPilot(test): test.execute(test_start=lambda: input("Serial: ")) # Measurements, limits, units, phases, attachments # all logged automatically. ``` ### pytest: Python SDK ```python filename="comparison/tofupilot_pytest.py" from tofupilot import TofuPilotClient client = TofuPilotClient() # After collecting results, create a run client.create_run( procedure_id="FCT-001", unit_under_test={"serial_number": "SN-001", "part_number": "PCBA-100"}, run_passed=True, steps=[ { "name": "test_power_rail", "step_passed": True, "measurements": [ { "name": "rail_3v3", "value": 3.31, "unit": "V", "lower_limit": 3.2, "upper_limit": 3.4, }, ], }, ], ) ``` More code, but you get the same analytics in TofuPilot: FPY, Cpk, control charts, failure Pareto, traceability. ## Decision Matrix Answer these questions: | Question | If yes → | If no → | |----------|----------|---------| | Are you running production/manufacturing tests? | OpenHTF | pytest | | Do operators interact with the test station? | OpenHTF | pytest | | Do you need structured measurement data (limits, units)? | OpenHTF | pytest | | Do you need parallel DUT testing? | pytest | Either | | Is your team new to both frameworks? | pytest (easier to learn) | N/A | | Do you need a rich plugin ecosystem? | pytest | Either | | Are you doing firmware CI testing? | pytest | N/A | If you answered "yes" to the first three questions, start with OpenHTF. For everything else, pytest is the safer bet. Both work with TofuPilot. ### What Is Test Data Management URL: https://www.tofupilot.com/guides/what-is-test-data-management-with-tofupilot Learn what test data management is, why spreadsheets fail at scale, and how TofuPilot structures your electronics test data automatically. Test data management (TDM) is the practice of collecting, storing, and analyzing every measurement from your production test stations in a single structured system. Without it, test results end up scattered across CSVs, local databases, and shared drives, and you lose the ability to trace failures, track yields, or prove compliance. ## The Problem with Spreadsheets and CSVs Most hardware teams start with CSV exports or shared spreadsheets. This works for a few units on one station. It breaks down fast. Common failure modes: - **No standard schema.** Each station logs different columns. Merging data requires manual cleanup every time. - **No unit traceability.** You can't look up the full test history for a single serial number without searching multiple files. - **No real-time visibility.** By the time someone opens a spreadsheet, the data is stale. Yield drops go unnoticed for hours or days. - **No audit trail.** Overwritten cells, deleted rows, renamed files. Regulators and customers won't accept this as evidence. At 100 units per day across two stations, you're already generating thousands of rows per week. At 10,000 units per day, spreadsheets aren't just inconvenient. They're a liability. ## What Structured Test Data Looks Like A proper TDM system stores every test run with a consistent structure: | Field | Example | |-------|---------| | Serial number | `SN-2026-00421` | | Part number | `PCB-RevC-Main` | | Station | `Station-3-Final` | | Outcome | PASS / FAIL | | Measurements | Voltage: 3.31V (limit: 3.0–3.6V) | | Timestamp | 2026-03-12T14:32:00Z | | Operator | Line 2, Shift A | | Attachments | Waveform capture, log file | Every measurement carries its name, value, unit, and limits. Every run links to a unit, a station, and a point in time. This structure is what makes FPY calculations, Cpk analysis, and failure Pareto charts possible. ## How TofuPilot Captures Test Data Automatically If you're using OpenHTF, TofuPilot plugs directly into your test script. You don't need to build a logging layer or manage database connections. Write your test, add the TofuPilot output, and every run uploads automatically. ```python filename="voltage_test.py" import openhtf as htf from openhtf.util import units from tofupilot.openhtf import TofuPilot @htf.measures( htf.Measurement("output_voltage") .with_units(units.VOLT) .in_range(minimum=3.0, maximum=3.6), htf.Measurement("current_draw") .with_units(units.AMPERE) .in_range(minimum=0.1, maximum=0.5), ) def test_power_rail(test): test.measurements.output_voltage = 3.31 test.measurements.current_draw = 0.25 def main(): test = htf.Test(test_power_rail) with TofuPilot(test): test.execute(test_start=lambda: "SN-2026-00421") if __name__ == "__main__": main() ``` Each run produces a structured record with the serial number, measurements, limits, units, and pass/fail outcome. TofuPilot stores it all and links it to the unit's full history. ## What You Get from Structured Data Once your test data flows into TofuPilot, the analytics dashboard gives you immediate visibility without writing a single query: - **First pass yield** trends per station, per product, per time window - **Cpk and control charts** for any measurement, so you catch drift before it causes failures - **Failure Pareto** showing which test phases fail most often, ranked by frequency - **Unit history** for any serial number, showing every test run from EVT through PVT and production - **Measurement histograms** showing distribution against limits You don't need to build dashboards or write scripts to compute these metrics. The structured data makes them automatic. ## TDM vs. MES vs. Custom Databases TDM isn't an MES. A manufacturing execution system tracks work orders, routing, and inventory. TDM focuses specifically on test results, measurements, and quality metrics. Building a custom database works until you need analytics, access control, or integrations. Most teams that start with PostgreSQL and a few Python scripts spend months maintaining infrastructure instead of improving their tests. TofuPilot sits between these approaches. It handles the storage, structure, and analytics so your test engineers can focus on writing better tests. ### PCBA Functional Testing: A Complete Guide URL: https://www.tofupilot.com/guides/pcba-functional-testing-a-complete-guide-with-python-and-tofupilot A complete guide to PCBA functional testing (FCT) with Python, OpenHTF, and TofuPilot, covering power rails, communication, current draw, and traceability. PCBA functional testing (FCT) verifies that an assembled circuit board works as designed. It's the last test before the board ships. Unlike ICT (in-circuit test), which checks individual components, FCT tests the board as a system: power rails, communication interfaces, firmware, and current draw. This guide shows you how to build a production FCT with Python, OpenHTF, and TofuPilot. ## PCBA Test Methods Compared | Method | What It Tests | When to Use | Equipment | |--------|-------------|-------------|-----------| | **ICT** (In-Circuit Test) | Individual components: resistors, capacitors, ICs | High volume, after SMT reflow | Bed-of-nails fixture, dedicated ICT tester | | **Flying Probe** | Same as ICT, no fixture needed | Low volume, prototypes | Flying probe machine | | **FCT** (Functional Test) | Board-level behavior: power, comms, firmware | Every board before shipping | Fixture + instruments + test script | | **Boundary Scan** (JTAG) | IC connections, digital logic | Complex BGA boards | JTAG adapter | | **AOI** (Automated Optical) | Solder joints, component placement | After reflow, before ICT/FCT | AOI machine | Most production lines run AOI after reflow, then FCT before packaging. ICT is optional and cost-effective only at high volumes (10K+ boards/year). ## FCT Test Coverage A typical FCT checks these categories: | Category | What to Test | Typical Measurements | |----------|-------------|---------------------| | **Power** | All voltage rails, current draw | 3.3V, 5V, 1.8V rails; idle and active current | | **Communication** | UART, SPI, I2C, USB, Ethernet | Firmware version query, self-test response | | **Digital I/O** | GPIO, LED, buttons | LED state, button response, logic levels | | **Analog** | ADC readings, DAC output | Calibrated voltage, sensor readings | | **Passive components** | Pull-up/down resistors | Resistance measurement (power off) | | **Firmware** | Version, self-test, flash integrity | Version string, CRC check | ## Step 1: Define Your Test Fixture A test fixture connects the DUT (device under test) to your instruments. For FCT, you typically need: | Instrument | Purpose | Connection | |-----------|---------|------------| | Bench power supply | Power the DUT | Banana plugs to fixture | | Multimeter (DMM) | Measure voltage, current, resistance | Test probes to fixture | | UART adapter | Communicate with DUT firmware | USB-to-serial to fixture | | Optional: oscilloscope | Verify signal integrity | Probes to test points | ## Step 2: Create Instrument Plugs Each instrument gets an OpenHTF Plug. The plug handles connection setup and teardown. Plugs are injected into test phases using the `@htf.plug()` decorator. ```python filename="fct/plugs.py" import openhtf as htf from openhtf.plugs import BasePlug class PowerSupplyPlug(BasePlug): """Bench power supply control.""" def setUp(self): self.output_on = False # Replace with real instrument connection (e.g., PyVISA) def set_voltage(self, channel: int, voltage: float): """Set output voltage on a channel.""" pass # Replace: psu.write(f":INST:SEL CH{channel}"); psu.write(f":VOLT {voltage}") def set_current_limit(self, channel: int, current: float): """Set current limit on a channel.""" pass # Replace: psu.write(f":CURR {current}") def enable_output(self): self.output_on = True # Replace: psu.write(":OUTP ON") def disable_output(self): self.output_on = False # Replace: psu.write(":OUTP OFF") def tearDown(self): self.disable_output() class MultimeterPlug(BasePlug): """DMM for voltage, current, and resistance measurements.""" def setUp(self): pass # Replace with PyVISA connection def measure_voltage(self) -> float: return 3.31 # Replace: float(dmm.query(":MEAS:VOLT:DC?")) def measure_current(self) -> float: return 0.12 # Replace: float(dmm.query(":MEAS:CURR:DC?")) def measure_resistance(self) -> float: return 4720.0 # Replace: float(dmm.query(":MEAS:RES?")) def tearDown(self): pass class UartPlug(BasePlug): """UART interface for DUT communication.""" def setUp(self): pass # Replace: serial.Serial("/dev/ttyUSB0", 115200) def send_command(self, cmd: str) -> str: if cmd == "AT+VERSION?": return "2.1.0" # Replace with real serial read if cmd == "AT+STATUS?": return "OK" return "" def tearDown(self): pass ``` ## Step 3: Write the Power Rail Test This phase powers the DUT and measures all voltage rails. ```python filename="fct/test_power.py" import openhtf as htf from openhtf.util import units @htf.measures( htf.Measurement("rail_3v3") .in_range(3.2, 3.4) .with_units(units.VOLT) .doc("3.3V supply rail"), htf.Measurement("rail_5v0") .in_range(4.8, 5.2) .with_units(units.VOLT) .doc("5.0V supply rail"), htf.Measurement("rail_1v8") .in_range(1.7, 1.9) .with_units(units.VOLT) .doc("1.8V core rail"), ) @htf.plug(psu=PowerSupplyPlug, dmm=MultimeterPlug) def test_power_rails(test, psu, dmm): """Power the DUT and verify all voltage rails.""" psu.set_voltage(1, 12.0) psu.set_current_limit(1, 1.0) psu.enable_output() test.measurements.rail_3v3 = dmm.measure_voltage() test.measurements.rail_5v0 = dmm.measure_voltage() test.measurements.rail_1v8 = dmm.measure_voltage() ``` ## Step 4: Write the Communication Test Verify the DUT firmware responds correctly over UART. ```python filename="fct/test_comms.py" import openhtf as htf @htf.measures( htf.Measurement("firmware_version") .equals("2.1.0") .doc("Firmware version string"), htf.Measurement("self_test_status") .equals("OK") .doc("Board self-test result"), ) @htf.plug(uart=UartPlug) def test_communication(test, uart): """Query firmware version and self-test status.""" test.measurements.firmware_version = uart.send_command("AT+VERSION?") test.measurements.self_test_status = uart.send_command("AT+STATUS?") ``` ## Step 5: Write the Current Draw Test Excessive current draw usually means a short or damaged component. ```python filename="fct/test_current.py" import openhtf as htf from openhtf.util import units @htf.measures( htf.Measurement("idle_current") .in_range(0.05, 0.20) .with_units(units.AMPERE) .doc("Board idle current consumption"), ) @htf.plug(dmm=MultimeterPlug) def test_current_draw(test, dmm): """Measure idle current draw.""" test.measurements.idle_current = dmm.measure_current() ``` ## Step 6: Write the Passive Component Test Check pullup resistors with the DUT powered off. Wrong resistor values cause intermittent communication failures that are hard to catch any other way. ```python filename="fct/test_passives.py" import openhtf as htf from openhtf.util import units @htf.measures( htf.Measurement("i2c_pullup") .in_range(4500, 5100) .with_units(units.OHM) .doc("I2C SDA pullup resistance"), ) @htf.plug(dmm=MultimeterPlug) def test_pullup_resistors(test, dmm): """Verify I2C pullup resistor values (power off).""" test.measurements.i2c_pullup = dmm.measure_resistance() ``` ## Step 7: Assemble and Run Connect all phases into a test with TofuPilot for production traceability. ```python filename="fct/main.py" import openhtf as htf from tofupilot.openhtf import TofuPilot def main(): test = htf.Test( test_power_rails, test_communication, test_current_draw, test_pullup_resistors, procedure_id="PCBA-FCT-001", part_number="PCBA-200", ) with TofuPilot(test): test.execute(test_start=lambda: input("Scan serial number: ")) if __name__ == "__main__": main() ``` Every measurement flows into TofuPilot with its name, value, limits, units, and pass/fail status. You get FPY, Cpk, and control charts per measurement without extra code. ## Design for Testability (DFT) Checklist Good FCT coverage starts at PCB design. Follow these guidelines: | DFT Rule | Why | Impact on FCT | |----------|-----|---------------| | Add test points to all voltage rails | DMM access without probing ICs | Direct voltage measurement | | Break out UART/SPI/I2C to header | Communication test without bed-of-nails | Firmware verification | | Add a power-on LED | Visual sanity check | Boolean measurement | | Include a self-test in firmware | Board can verify its own peripherals | Single command validates multiple subsystems | | Label test points on silkscreen | Operators can probe manually if needed | Reduces fixture complexity | | Route test points to board edge | Pogo pin access in fixture | Faster fixture build | ## Common FCT Failure Modes | Failure | Typical Cause | How to Detect | |---------|--------------|---------------| | Voltage rail out of spec | Wrong resistor value in regulator divider, cold solder joint | Measure rail voltage with limits | | Excessive current draw | Solder bridge, damaged IC | Measure idle and active current | | Communication timeout | Missing pullup, wrong baud rate, unflashed IC | Query firmware, check response time | | Wrong firmware version | Flashing error, wrong binary | Query version string, compare to expected | | Intermittent failure | Marginal solder joint, loose connector | Run test multiple times, check measurement variance | ### What Is First Article Inspection (FAI) URL: https://www.tofupilot.com/guides/what-is-fai-with-tofupilot First article inspection (FAI) verifies the first production unit meets all specifications. Learn how FAI works and how to track it with TofuPilot. # What Is FAI with TofuPilot First article inspection (FAI) is a detailed verification of the first production unit to confirm that the manufacturing process can produce parts that meet all design specifications. It's required by aerospace (AS9102), automotive (PPAP), and medical device standards. This guide covers what FAI involves, when it's required, and how to track FAI results with TofuPilot. ## What FAI Verifies FAI checks every characteristic on the drawing and specification, not just the critical ones. It's a full accounting of: | What's Checked | Example | |---------------|---------| | Dimensional characteristics | All dimensions on the drawing | | Material certifications | Raw material CoC, RoHS compliance | | Process records | Solder profile, torque records, adhesive cure | | Functional test results | All measurements against specification | | Visual and cosmetic | Surface finish, marking, label placement | | Traceability | Serial numbers, lot codes, date codes | FAI is not sampling. It's a complete inspection of one unit (or a set of units) against every requirement. ## When FAI Is Required | Trigger | Example | |---------|---------| | New product introduction | First production run of a new design | | Design change | Engineering change order (ECO) affecting form, fit, or function | | Process change | New supplier, new tooling, new factory location | | Extended production break | No production for 2+ years (per AS9102) | | Corrective action | After a nonconformance that affected product quality | ## FAI Documentation (AS9102) AS9102 defines three standard FAI report forms: | Form | Content | |------|---------| | Form 1: Part Number Accountability | Part number, name, revision, serial number, materials | | Form 2: Product Accountability (Raw Material, Special Process, Functional Test) | Material certs, process records, test data | | Form 3: Characteristic Accountability | Every dimension and characteristic with actual measured value | Form 3 is the most labor-intensive. Every dimension on the drawing gets a balloon number, and the corresponding measured value goes in the FAI report. ## Prerequisites - Python 3.10+ - OpenHTF installed (`pip install openhtf`) - TofuPilot Python SDK installed (`pip install tofupilot`) ## Step 1: Define FAI Test Phases The functional test portion of FAI runs the same test procedure as production, but you record every measurement, not just pass/fail. ```python filename="fai_test.py" import openhtf as htf from openhtf.util import units @htf.measures( htf.Measurement("output_voltage_V") .in_range(minimum=4.9, maximum=5.1) .with_units(units.VOLT), htf.Measurement("ripple_voltage_mV") .in_range(maximum=50) .with_units(units.MILLIVOLT), htf.Measurement("efficiency_percent") .in_range(minimum=90) .with_units(units.PERCENT), htf.Measurement("leakage_current_uA") .in_range(maximum=500) .with_units(units.MICROAMPERE), ) def phase_fai_electrical(test): """FAI Form 2: All electrical characteristics.""" test.measurements.output_voltage_V = 5.02 test.measurements.ripple_voltage_mV = 18.3 test.measurements.efficiency_percent = 93.7 test.measurements.leakage_current_uA = 65.0 @htf.measures( htf.Measurement("firmware_version").equals("2.1.0"), htf.Measurement("boot_time_ms") .in_range(maximum=2000) .with_units(units.MILLISECOND), htf.Measurement("self_test").equals("PASS"), ) def phase_fai_functional(test): """FAI Form 2: Functional characteristics.""" test.measurements.firmware_version = "2.1.0" test.measurements.boot_time_ms = 1120 test.measurements.self_test = "PASS" ``` ## Step 2: Log FAI Results to TofuPilot The FAI test runs once on the first article. TofuPilot stores every measurement with its limit, unit, and result. ```python filename="fai_test.py" from tofupilot.openhtf import TofuPilot test = htf.Test( phase_fai_electrical, phase_fai_functional, ) with TofuPilot(test): test.execute(test_start=lambda: input("Scan first article serial: ")) ``` ## Step 3: Generate the FAI Report TofuPilot stores the complete test record for the first article. Open the unit's test history to get: - **All measurements** with actual values against specification limits - **Pass/fail status** for each characteristic - **Timestamps** for audit trail - **Test reports** exportable for AS9102 Form 2 documentation This data feeds directly into the FAI report. Instead of transcribing measurements from a bench multimeter into a spreadsheet, the test data is already structured and auditable. ## FAI vs Production Test | Aspect | FAI | Production Test | |--------|-----|-----------------| | Scope | Every characteristic on the drawing | Critical functional characteristics | | Sample | First unit (or first from each cavity/tool) | Every unit | | Documentation | Formal FAI report (AS9102 forms) | Internal test records | | Frequency | Once per trigger event | Every unit, every run | | Goal | Prove the process can make a conforming part | Verify each unit conforms | FAI proves the process is capable. Production test proves each unit is conforming. Both are necessary. TofuPilot tracks both using the same infrastructure. ### What Is HALT and HASS Testing URL: https://www.tofupilot.com/guides/what-is-halt-and-hass-testing-with-tofupilot HALT finds design limits, HASS screens production units. Learn how both work, when to use each, and how to track results with TofuPilot. # What Is HALT and HASS Testing with TofuPilot HALT (Highly Accelerated Life Test) pushes a product beyond its design limits to find failure modes early. HASS (Highly Accelerated Stress Screen) applies controlled stress during production to catch latent defects before they reach customers. This guide covers how both methods work, when to use each, and how to track stress test results with TofuPilot. ## HALT: Find the Weak Spots HALT is a design-phase test. You put prototype units in a chamber and ramp temperature and vibration until something breaks. The goal is not to simulate field conditions. The goal is to find the fundamental limits of your design. A typical HALT profile includes: | Step | What Happens | |------|-------------| | Cold step stress | Step temperature down in increments until failure | | Hot step stress | Step temperature up in increments until failure | | Rapid thermal transitions | Cycle between cold and hot limits at max chamber rate | | Vibration step stress | Increase random vibration in steps until failure | | Combined environment | Apply thermal cycling and vibration simultaneously | Each time a failure occurs, you analyze the root cause, fix it if possible, and continue. HALT is iterative. A single HALT run typically uncovers 3 to 8 distinct failure modes. ### HALT Output | Result | What It Tells You | |--------|------------------| | Operating limits | Temperature and vibration where the product stops working but recovers | | Destruct limits | Where permanent damage occurs | | Design margin | Gap between operating limits and product specification | | Failure modes | Ranked list of weaknesses (solder joints, connectors, components) | HALT runs on 5 to 15 units during EVT or DVT. It's qualitative, not statistical. You're looking for failure modes, not predicting field life. ## HASS: Screen Every Production Unit HASS applies stress levels between the product's specification and the destruct limits found in HALT. It runs on every production unit (or a defined sample) to precipitate latent defects that would cause early field failures. | Aspect | HALT | HASS | |--------|------|------| | When | Design phase (EVT/DVT) | Production | | Sample size | 5-15 units | Every unit or sample | | Stress level | Beyond spec, until failure | Below destruct limits | | Goal | Find design weaknesses | Screen manufacturing defects | | Duration | 1-2 weeks | Minutes to hours per unit | | Output | Failure modes, design margins | Pass/fail per unit | HASS stresses are derived from HALT. If HALT found a destruct limit at -60C, a HASS screen might cycle to -40C. The stress is high enough to catch latent defects but low enough to avoid damaging good units. ### Proof of Screen (POS) Before running HASS on production units, you validate the screen itself: 1. Run the HASS profile on units known to be good. Verify zero failures. 2. Seed units with known defects. Verify the screen catches them. 3. Document the detection rate. If the screen misses seeded defects, increase stress. ## Prerequisites - Python 3.10+ - OpenHTF installed (`pip install openhtf`) - TofuPilot Python SDK installed (`pip install tofupilot`) ## Step 1: Define Functional Check Phases Both HALT and HASS require functional checks between stress cycles. These detect when a unit degrades or fails. ```python filename="stress_check.py" import openhtf as htf from openhtf.util import units @htf.measures( htf.Measurement("output_voltage_V") .in_range( minimum=3.0, maximum=3.6, marginal_minimum=3.1, marginal_maximum=3.5, ) .with_units(units.VOLT), htf.Measurement("current_draw_mA") .in_range(minimum=40, maximum=65) .with_units(units.MILLIAMPERE), ) def phase_electrical_check(test): """Verify core electrical parameters after stress cycle.""" test.measurements.output_voltage_V = 3.29 test.measurements.current_draw_mA = 51.3 @htf.measures( htf.Measurement("comms_loopback").equals("PASS"), ) def phase_comms_check(test): """Verify communication interfaces still respond.""" test.measurements.comms_loopback = "PASS" ``` ## Step 2: Log Each Cycle to TofuPilot Run the functional check after each stress step. Each execution creates a new test run linked to the same serial number, building a degradation timeline. ```python filename="stress_check.py" from tofupilot.openhtf import TofuPilot test = htf.Test( phase_electrical_check, phase_comms_check, ) with TofuPilot(test): test.execute(test_start=lambda: input("Scan unit serial: ")) ``` ## Step 3: Track Degradation in TofuPilot TofuPilot tracks every run per serial number. Open the unit history to see: - **Measurement trends** across stress cycles (voltage drift, current increase) - **Marginal results** flagged before hard failures - **Control charts** showing when a parameter trends toward a limit - **Failure analysis** identifying which stress step causes the most failures For HASS, the pass/fail result of the production screen feeds directly into yield tracking. TofuPilot shows HASS yield alongside your other production test stages. ## When to Use HALT vs HASS vs ESS | Method | Phase | Stress Level | Sample | Best For | |--------|-------|-------------|--------|----------| | HALT | EVT/DVT | Beyond spec | 5-15 units | Finding design weaknesses | | HASS | Production | Below destruct | Every unit | Screening latent defects | | ESS | Production | Within spec | Every unit | Precipitating infant mortality | | ALT | DVT | Above spec | 20-50 units | Predicting field life | | ORT | Production | Per spec | Periodic sample | Monitoring ongoing reliability | HALT and HASS work as a pair. HALT defines the design margins. HASS uses those margins to build a production screen. Without HALT data, HASS stress levels are guesswork. ### HIL Testing with Python URL: https://www.tofupilot.com/guides/hardware-in-the-loop-testing-with-python-a-practical-guide Learn how to set up HIL testing for embedded systems with Python, OpenHTF, and TofuPilot, including GPIO, ADC, PWM, and communication interface testing. Hardware-in-the-loop (HIL) testing validates embedded firmware by stimulating real hardware inputs and measuring real outputs. Unlike pure simulation, HIL tests run on the actual target hardware with real peripherals. This catches bugs that simulation misses: timing issues, ADC noise, interrupt conflicts, and peripheral initialization problems. This guide shows you how to build a Python-based HIL test system with OpenHTF and TofuPilot. ## HIL vs SIL vs MIL | Method | Hardware | Software | Fidelity | Speed | Cost | |--------|----------|----------|----------|-------|------| | **MIL** (Model-in-the-Loop) | None | Simulated plant + controller | Low | Fast (seconds) | Low | | **SIL** (Software-in-the-Loop) | None | Real controller code on host | Medium | Fast (seconds) | Low | | **HIL** (Hardware-in-the-Loop) | Real target board | Real firmware | High | Real-time | Medium | | **Physical test** | Real target + real plant | Real firmware | Highest | Real-time | High | HIL sits between SIL and physical testing. You test real firmware on real hardware, but simulate the environment (sensors, actuators, external systems). ## When to Use HIL Testing - **Firmware regression testing.** Every firmware build gets the same physical tests. Catches regressions that unit tests miss. - **Peripheral validation.** GPIO, ADC, DAC, PWM, UART, SPI, I2C. Real silicon behaves differently than simulation models. - **Safety-critical systems.** Automotive, medical, aerospace require HIL testing for certification (ISO 26262, IEC 62304). - **Pre-production validation.** Validate firmware on real hardware before committing to production. ## HIL Test Architecture A typical Python HIL setup: | Component | Role | Example | |-----------|------|---------| | Target board (DUT) | Runs firmware under test | STM32 dev board, ESP32, Raspberry Pi Pico | | Test host | Runs Python test scripts | PC or Raspberry Pi | | DAQ / GPIO adapter | Stimulate inputs, read outputs | Arduino, LabJack, NI DAQ | | Power supply | Controlled power to DUT | Bench PSU (programmable) | | Serial/debug | Flash firmware, send commands | USB-to-serial, J-Link, ST-Link | | Instruments | Precision measurements | DMM, oscilloscope | ## Step 1: Create Hardware Interface Plugs Each piece of test hardware gets an OpenHTF Plug. This keeps the test phases clean and instrument-agnostic. ```python filename="hil/plugs.py" import openhtf as htf from openhtf.plugs import BasePlug class GpioPlug(BasePlug): """GPIO interface for stimulating DUT inputs and reading outputs. Replace with real GPIO adapter (e.g., LabJack, Arduino, RPi.GPIO). """ def setUp(self): self._pin_states = {} def set_pin(self, pin: int, state: bool): """Set a GPIO pin on the test fixture (DUT input).""" self._pin_states[pin] = state def read_pin(self, pin: int) -> bool: """Read a GPIO pin from the DUT (DUT output).""" return self._pin_states.get(pin, False) def tearDown(self): self._pin_states.clear() class AdcPlug(BasePlug): """ADC interface for reading analog outputs from DUT. Replace with real DAQ (e.g., NI DAQmx, LabJack, ADS1115). """ def setUp(self): self._channels = {0: 1.65, 1: 2.50, 2: 0.33} def read_channel(self, channel: int) -> float: """Read an ADC channel voltage.""" return self._channels.get(channel, 0.0) def tearDown(self): pass class PwmPlug(BasePlug): """PWM measurement interface. Replace with real frequency counter or oscilloscope. """ def setUp(self): pass def measure_frequency(self) -> float: """Measure PWM output frequency in Hz.""" return 1000.0 def measure_duty_cycle(self) -> float: """Measure PWM duty cycle in percent.""" return 50.0 def tearDown(self): pass ``` ## Step 2: Write GPIO Tests The simplest HIL test: set an input, check the firmware drives the correct output. ```python filename="hil/test_gpio.py" import openhtf as htf @htf.measures( htf.Measurement("gpio_loopback") .equals(True) .doc("GPIO pin loopback: set input, verify output follows"), ) @htf.plug(gpio=GpioPlug) def test_gpio_loopback(test, gpio): """Verify firmware routes GPIO input to output correctly.""" gpio.set_pin(1, True) # Stimulate DUT input readback = gpio.read_pin(1) # Read DUT output test.measurements.gpio_loopback = readback ``` ## Step 3: Write ADC/Sensor Tests Simulate sensor inputs and verify the DUT reads them correctly. ```python filename="hil/test_adc.py" import openhtf as htf from openhtf.util import units @htf.measures( htf.Measurement("adc_channel_0") .in_range(1.5, 1.8) .with_units(units.VOLT) .doc("ADC ch0: 1.65V reference voltage"), htf.Measurement("adc_channel_1") .in_range(2.3, 2.7) .with_units(units.VOLT) .doc("ADC ch1: 2.5V sensor output"), ) @htf.plug(adc=AdcPlug) def test_adc_readings(test, adc): """Verify ADC channels read expected voltages.""" test.measurements.adc_channel_0 = adc.read_channel(0) test.measurements.adc_channel_1 = adc.read_channel(1) ``` ## Step 4: Write PWM Tests Verify the firmware generates the correct PWM signal (frequency and duty cycle). ```python filename="hil/test_pwm.py" import openhtf as htf from openhtf.util import units @htf.measures( htf.Measurement("pwm_frequency") .in_range(950, 1050) .with_units(units.HERTZ) .doc("PWM output frequency (target: 1kHz)"), htf.Measurement("pwm_duty_cycle") .in_range(48, 52) .doc("PWM duty cycle percentage (target: 50%)"), ) @htf.plug(pwm=PwmPlug) def test_pwm_output(test, pwm): """Verify PWM output frequency and duty cycle.""" test.measurements.pwm_frequency = pwm.measure_frequency() test.measurements.pwm_duty_cycle = pwm.measure_duty_cycle() ``` ## Step 5: Assemble the HIL Test Connect all phases with TofuPilot for tracking firmware version regressions over time. ```python filename="hil/main.py" import openhtf as htf from tofupilot.openhtf import TofuPilot def main(): test = htf.Test( test_gpio_loopback, test_adc_readings, test_pwm_output, procedure_id="HIL-001", part_number="ECU-100", ) with TofuPilot(test): test.execute(test_start=lambda: input("Firmware version: ")) if __name__ == "__main__": main() ``` Use the serial number prompt to enter the firmware version. This lets TofuPilot track which firmware builds pass or fail each test, building a regression history. ## HIL in CI/CD The real power of HIL testing comes from running it automatically on every firmware build. | CI/CD Step | Action | |-----------|--------| | 1. Code push | Developer pushes firmware change | | 2. Build | CI compiles firmware binary | | 3. Flash | CI flashes binary to target board (via J-Link/ST-Link) | | 4. HIL test | CI runs OpenHTF test script | | 5. Results | TofuPilot logs pass/fail per measurement | | 6. Gate | CI fails the build if any measurement is out of spec | This requires a dedicated test station connected to your CI runner. The test station has the target board, GPIO adapter, DAQ, and instruments. The CI runner (Jenkins, GitHub Actions self-hosted runner, etc.) triggers the test after each build. ## Common HIL Test Categories | Category | What to Test | Measurements | |----------|-------------|-------------| | **GPIO** | Input/output routing, interrupt response | Pin state, response time | | **ADC** | Sensor reading accuracy, noise | Voltage values, noise floor | | **DAC** | Output voltage accuracy | Voltage values | | **PWM** | Frequency, duty cycle, resolution | Frequency, duty cycle | | **Communication** | UART, SPI, I2C, CAN data integrity | Bit error rate, response time | | **Timing** | Interrupt latency, task scheduling | Response time in microseconds | | **Power** | Sleep mode current, wake-up time | Current draw, wake-up latency | | **Watchdog** | Recovery from hang | Reset detection, recovery time | ## HIL vs Production FCT HIL and FCT test different things at different stages: | | HIL | FCT | |--|-----|-----| | **When** | Development, every firmware build | Production, every manufactured board | | **What** | Firmware behavior on real hardware | Board assembly and system function | | **DUT** | Dev board or prototype | Production board | | **Changes between tests** | Firmware (code changes) | Hardware (manufacturing variation) | | **Failure means** | Firmware bug (code fix) | Assembly defect (rework or scrap) | | **Run frequency** | Every CI build | Every unit manufactured | | **TofuPilot use** | Track regressions across firmware versions | Track FPY, Cpk, control charts | Many teams use the same OpenHTF test framework for both HIL and FCT. The plugs change (dev board GPIO vs. production fixture), but the test structure is the same. ### TestStand Alternatives for Manufacturing Test URL: https://www.tofupilot.com/guides/teststand-alternatives-with-tofupilot Compare open-source and commercial alternatives to NI TestStand for manufacturing test, with code examples, cost analysis, and criteria for OpenHTF. NI TestStand costs $4,310/seat/year and locks you into Windows. If you're evaluating alternatives, your real options are OpenHTF, pytest, and OpenTAP. Each targets a different use case. This guide compares all three against TestStand with code examples, cost breakdowns, and a decision framework so you can pick the right one. ## Why Teams Look for Alternatives TestStand renewal invoices trigger this search every year. But cost isn't the only reason teams migrate. These are the pain points that come up most in NI forums and engineering communities: | Pain Point | Impact | |---|---| | **$4,310/seat/year** + deployment licenses per station | Scales badly for multi-site operations | | **Windows only** | Can't run test stations on Linux (cheaper, more stable) | | **Binary .seq files** | Can't diff, can't code review, can't merge in Git | | **Complex database schema** | 5-6 table JOINs for a simple measurement query | | **No built-in analytics** | FPY, Cpk, control charts require custom SQL or WATS | | **Specialized hiring** | Need TestStand-trained engineers, smaller talent pool | | **Process Model lock-in** | Customizing reports, DB logging, or UUT flow means editing NI's Process Model | NI added Git support and CI/CD licensing in TestStand 2025 Q2, but the sequence files are still binary. The Git pane lets you commit .seq files, but you can't diff the contents or review changes in a pull request. It's version tracking, not version control. ## The Alternatives ### OpenHTF (Google, Free) OpenHTF is a Python framework built by Google specifically for manufacturing test. It has structured measurements with limits and units, serial number prompts, and phase-based test sequencing. It's the closest direct replacement for TestStand. ```python filename="openhtf_example.py" import openhtf as htf from openhtf.util import units from tofupilot.openhtf import TofuPilot @htf.measures( htf.Measurement("rail_3v3") .in_range(3.2, 3.4) .with_units(units.VOLT), htf.Measurement("current_draw") .in_range(0.1, 0.5) .with_units(units.AMPERE), ) def functional_test(test): test.measurements.rail_3v3 = 3.31 test.measurements.current_draw = 0.24 def main(): test = htf.Test( functional_test, procedure_id="FCT-001", part_number="PCBA-200", ) with TofuPilot(test): test.execute(test_start=lambda: input("Scan serial: ")) if __name__ == "__main__": main() ``` **Strengths:** Measurements are first-class objects with names, values, limits, and units. TofuPilot integration is one line. Phases run in order with automatic pass/fail evaluation. Plugs manage instrument lifecycle (setUp/tearDown). **Limitations:** Parallel DUT support is limited. Smaller community (~640 GitHub stars). No GUI editor for non-programmers. **Best for:** Production FCT, teams that write Python, companies that want structured test data without a database. ### pytest (Community, Free) pytest is the most popular Python testing framework. It wasn't built for hardware testing, but its flexibility and massive ecosystem make it a viable option for R&D validation and firmware CI. ```python filename="pytest_example.py" import pytest from tofupilot import TofuPilotClient @pytest.fixture def client(): return TofuPilotClient() def test_power_rail(client): voltage = 3.31 # Replace with instrument read assert 3.2 <= voltage <= 3.4, f"3.3V rail out of range: {voltage}V" client.create_run( procedure_id="FCT-001", unit_under_test={"serial_number": "SN-5001"}, run_passed=True, steps=[{ "name": "power_rail", "step_passed": True, "measurements": [{ "name": "rail_3v3", "measured_value": voltage, "unit": "V", "lower_limit": 3.2, "upper_limit": 3.4, }], }], ) ``` **Strengths:** Every Python developer knows it. Massive plugin ecosystem (pytest-xdist for parallelism, pytest-html for reports). Fixtures are flexible. Native CI/CD integration. **Limitations:** Measurements are implicit (assert statements, not structured data). No built-in serial number handling. No operator UI. You write more boilerplate to get structured results into TofuPilot. **Best for:** R&D validation, firmware CI/CD, teams that already use pytest for software tests. ### OpenTAP (Keysight, Free) OpenTAP is an open-source test sequencer from Keysight. It uses C# as its primary language (Python plugin available) and has a GUI editor for building test plans visually. It's the closest to TestStand's workflow model. ```python filename="opentap_example.py" import opentap from opentap import attribute, display @display("Power Rail Test", "Measure 3.3V rail") class PowerRailStep(opentap.TestStep): lower_limit = attribute(default_value=3.2) upper_limit = attribute(default_value=3.4) def run(self): voltage = 3.31 # Replace with instrument read self.publish_result("rail_3v3", {"Voltage": voltage}) if voltage < self.lower_limit or voltage > self.upper_limit: self.upgrade_verdict(opentap.Verdict.Fail) ``` **Strengths:** GUI step editor for non-programmers. Plugin architecture for instruments and result listeners. Keysight instrument integration. Test plans can be edited without code. **Limitations:** C# is the primary language (Python is secondary). Smaller community (~200 GitHub stars). Linux support exists but Windows is the primary target. **Best for:** Teams with Keysight instrument stacks, organizations that need a GUI editor for test plan creation. ## Feature Comparison | Feature | TestStand | OpenHTF | pytest | OpenTAP | |---------|-----------|---------|--------|---------| | **License** | $4,310/seat/year | Free | Free | Free | | **Language** | LabVIEW, C, .NET, Python | Python | Python | C#, Python | | **Platform** | Windows only | Linux, macOS, Windows | Linux, macOS, Windows | Windows, Linux | | **Structured measurements** | Built-in | Built-in | Manual (assert) | Plugin-based | | **Serial number input** | Built-in | Built-in | Manual | Plugin | | **Operator UI** | Built-in | Built-in web UI | None | Built-in GUI editor | | **Version control** | Difficult (.seq binary) | Git native (.py) | Git native (.py) | Git (XML + code) | | **CI/CD** | Added 2025 Q2 (limited) | Native | Native | Possible | | **Instrument drivers** | NI VISA, IVI | PyVISA, pyserial | PyVISA, pyserial | Plugins (C#/Python) | | **Parallel DUT** | Native | Limited | Native (xdist) | Native | | **Database logging** | Built-in (complex schema) | TofuPilot (1 line) | TofuPilot (~10 lines) | REST API | | **Analytics** | Custom SQL or WATS | TofuPilot (automatic) | TofuPilot (automatic) | Custom | | **Community** | Large (NI forums) | Small (~640 stars) | Massive (11K+ stars) | Small (~200 stars) | | **Learning curve** | High | Medium | Low | Medium | ## Cost Comparison | Scenario | TestStand | OpenHTF | pytest | OpenTAP | |----------|-----------|---------|--------|---------| | 5 dev seats, 1 year | $21,550 | $0 | $0 | $0 | | 20 dev seats, 1 year | $86,200 | $0 | $0 | $0 | | 10 deployment stations | Extra licenses | Free | Free | Free | | Training | NI courses ($2K+) | openhtf.com (free) | docs.pytest.org (free) | opentap.io (free) | | OS per station | Windows license | Free (Linux) | Free (Linux) | Windows or Linux | | Test data analytics | WATS or custom | TofuPilot (free tier) | TofuPilot (free tier) | Custom | TestStand's total cost includes development licenses, deployment licenses for every station, Windows licenses, and often a WATS subscription for analytics. The open-source alternatives eliminate all of these. ## TestStand Python Integration vs Going Full Python TestStand 2025 Q2 added Python step types. You can call Python functions from TestStand sequences. But this isn't the same as going fully Python-native. | Aspect | Python inside TestStand | Full Python (OpenHTF/pytest) | |--------|------------------------|------------------------------| | Sequencing | TestStand (.seq file) | Python code (.py file) | | Limits/measurements | TestStand step types | OpenHTF Measurement or assert | | Version control | Binary .seq + diffable .py | All diffable .py | | Licensing | Still need TestStand seat | Free | | Debugging | TestStand debugger + Python debugger | One debugger (Python) | | CI/CD | TestStand CI runner | Standard Python CI | | Data logging | TestStand DB schema | TofuPilot | | Deployment | TestStand + Python on every station | Python only | Using Python inside TestStand gives you Python's ecosystem for instrument control and test logic. But you're still paying for TestStand licenses, managing binary sequence files, and dealing with the database schema. Going fully Python-native removes all of those constraints. ## TofuPilot Integration All alternatives work with TofuPilot for test data management and analytics: | Framework | Integration | Effort | |-----------|-------------|--------| | OpenHTF | `with TofuPilot(test):` | 1 line | | pytest | `TofuPilotClient().create_run(...)` | ~10 lines per test | | OpenTAP | REST API or Python SDK | Medium | | TestStand | REST API | Medium | TofuPilot replaces TestStand's database logging, WATS, and custom analytics queries. FPY, Cpk, control charts, failure Pareto, and serial number traceability are all automatic. Open the Analytics tab to see results for any procedure. ## Decision Framework ### Choose OpenHTF if: - You're building production FCT or end-of-line tests - Your team writes Python - You want structured measurements with limits and units as code - You need an operator serial number prompt - You want analytics (FPY, Cpk) without building a database ### Choose pytest if: - You're doing R&D validation or firmware CI - Your team already uses pytest for software tests - You need maximum flexibility and plugin ecosystem - You don't need a built-in operator UI - Speed of iteration matters more than structured measurement data ### Choose OpenTAP if: - You have a large Keysight instrument stack - Non-programmers need to edit test plans via GUI - Your team works in C# more than Python - You want a plugin-based architecture for test steps ### Stay on TestStand if: - You have deep NI PXI/CompactRIO hardware investment - Your team is trained on TestStand and productive - You're locked into NI support contracts - Migration risk outweighs the cost savings ## Migration Paths ### TestStand to OpenHTF The most common migration. Every TestStand concept has a direct OpenHTF equivalent: sequences become test scripts, steps become phases, Code Modules become plugs, the Process Model becomes TofuPilot. Typical timeline: 4-8 weeks per test procedure. Run both systems in parallel during the transition. ### TestStand to pytest Works well when you're also moving from production FCT to R&D validation workflows. pytest fixtures replace TestStand Code Modules. You lose structured measurements (assert-only) but gain CI/CD integration and the pytest plugin ecosystem. ### TestStand to OpenTAP Closest workflow match if your team relies on the GUI editor. OpenTAP's step editor is similar to TestStand's Sequence Editor. Plugin architecture maps to TestStand's adapter pattern. Main barrier: C# learning curve if your team is Python-focused. ### Gradual migration You don't have to switch everything at once. Start with one test procedure on one station. Run both systems on the same DUTs. Validate that measurements match. Move to the next procedure once confirmed. Keep TestStand as fallback until you're confident. ### Handle Test Failures and Retries URL: https://www.tofupilot.com/guides/how-to-handle-test-failures-and-retries-with-tofupilot Learn how to control OpenHTF phase failure behavior, implement retry logic, and ensure clean teardown in production tests with TofuPilot. When a measurement fails in OpenHTF, the default behavior is to mark the phase as failed and continue running subsequent phases. In production, you need deliberate control: when to stop early, when to retry, and how to preserve failure data in TofuPilot for analytics. ## Prerequisites - TofuPilot Python client installed: `pip install tofupilot` - OpenHTF installed: `pip install openhtf` - Basic familiarity with OpenHTF phases and measurements ## Phase Failure Behavior by Default OpenHTF phases return a `PhaseResult`. When a measurement fails, the phase is marked `FAIL` but execution continues unless you explicitly stop it. ```python filename="tests/voltage_check.py" import openhtf as htf from openhtf.util import units from tofupilot.openhtf import TofuPilot @htf.measures( htf.Measurement('rail_3v3_voltage').in_range(3.2, 3.4).with_units(units.VOLT), htf.Measurement('rail_5v_voltage').in_range(4.85, 5.15).with_units(units.VOLT), ) def check_power_rails(test): test.measurements.rail_3v3_voltage = 3.1 # fails: below minimum test.measurements.rail_5v_voltage = 5.02 # passes @htf.measures( htf.Measurement('uart_echo').equals('OK'), ) def check_uart(test): # This phase still runs even though check_power_rails failed test.measurements.uart_echo = 'OK' def main(): test = htf.Test(check_power_rails, check_uart) with TofuPilot(test): test.execute(test_start=lambda: 'SN-001') ``` Both phases run. The run is uploaded to TofuPilot as FAIL with the specific failing measurement recorded. ## Stop on First Failure For hardware tests where a failed power rail makes downstream tests meaningless, stop early using `PhaseResult.STOP`. ```python filename="tests/production_sequence.py" import openhtf as htf from openhtf.util import units from tofupilot.openhtf import TofuPilot @htf.measures( htf.Measurement('rail_3v3_voltage').in_range(3.2, 3.4).with_units(units.VOLT), ) def check_power_rails(test): test.measurements.rail_3v3_voltage = read_adc_channel(0) if not test.measurements.rail_3v3_voltage.is_pass: test.logger.error('Power rail out of spec, aborting sequence') return htf.PhaseResult.STOP @htf.measures( htf.Measurement('uart_echo').equals('OK'), ) def check_uart(test): response = send_uart_command('PING') test.measurements.uart_echo = response def main(): test = htf.Test(check_power_rails, check_uart) with TofuPilot(test): test.execute(test_start=lambda: 'SN-001') ``` When `check_power_rails` returns `PhaseResult.STOP`, `check_uart` is skipped. TofuPilot records the run as FAIL with `check_uart` phases marked as not executed. ## Retry a Phase on Failure Retries are useful for transient failures: communication timeouts, settling voltages, or flaky contacts. Use `PhaseResult.REPEAT` with a counter to avoid infinite loops. ```python filename="tests/comms_test.py" import openhtf as htf from openhtf.util import units from tofupilot.openhtf import TofuPilot MAX_RETRIES = 3 @htf.measures( htf.Measurement('i2c_device_present').equals(True), ) def check_i2c_device(test, retries=[0]): found = probe_i2c_address(0x48) test.measurements.i2c_device_present = found if not found and retries[0] < MAX_RETRIES: retries[0] += 1 test.logger.warning('I2C device not found, retry %d/%d', retries[0], MAX_RETRIES) return htf.PhaseResult.REPEAT retries[0] = 0 # reset for future runs @htf.measures( htf.Measurement('i2c_temperature').in_range(-10, 85).with_units(units.DEGREE_CELSIUS), ) def read_i2c_temperature(test): temp = read_temperature_sensor(0x48) test.measurements.i2c_temperature = temp def main(): test = htf.Test(check_i2c_device, read_i2c_temperature) with TofuPilot(test): test.execute(test_start=lambda: 'SN-002') ``` Using a mutable default argument (`retries=[0]`) persists the counter across `REPEAT` calls within the same phase execution. Reset it before returning to avoid stale state on the next DUT. ## Retry with Backoff Using a Plug For production lines with many fixtures, a plug keeps retry logic reusable and keeps phase code clean. ```python filename="plugs/comms_plug.py" import time import openhtf as htf from openhtf.plugs import BasePlug class CommPlug(BasePlug): def setUp(self): import serial self._port = serial.Serial('/dev/ttyUSB0', 115200, timeout=1) def send_with_retry(self, command, expected, attempts=3, delay=0.5): response = '' for attempt in range(attempts): self._port.write(f'{command}\n'.encode()) response = self._port.readline().decode().strip() if response == expected: return response, attempt + 1 time.sleep(delay) return response, attempts def tearDown(self): if self._port: self._port.close() ``` ```python filename="tests/serial_test.py" import openhtf as htf from tofupilot.openhtf import TofuPilot from plugs.comms_plug import CommPlug @htf.plug(comm=CommPlug) @htf.measures( htf.Measurement('firmware_version').matches_regex(r'v\d+\.\d+\.\d+'), htf.Measurement('version_retry_count').in_range(minimum=1), ) def check_firmware_version(test, comm): response, attempts = comm.send_with_retry('VERSION', expected='v2.1.0', attempts=3) test.measurements.firmware_version = response test.measurements.version_retry_count = attempts def main(): test = htf.Test(check_firmware_version) with TofuPilot(test): test.execute(test_start=lambda: 'SN-003') ``` Recording `version_retry_count` as a measurement lets TofuPilot surface retry frequency across your production fleet. A spike in retries on a specific station points to a fixture problem before it becomes a yield problem. ## Clean Teardown After Failures Plugs with a `tearDown` method are always called by OpenHTF, even when a phase fails or the test stops early. Use this to release hardware resources reliably. ```python filename="plugs/fixture_plug.py" import openhtf as htf from openhtf.plugs import BasePlug class FixturePlug(BasePlug): def setUp(self): self._power_on = False self._relay_closed = False def power_on_dut(self): enable_power_rail() self._power_on = True def close_test_relay(self): close_relay(1) self._relay_closed = True def tearDown(self): # Always runs, even on STOP or exception if self._relay_closed: open_relay(1) if self._power_on: disable_power_rail() ``` ```python filename="tests/powered_test.py" import openhtf as htf from openhtf.util import units from tofupilot.openhtf import TofuPilot from plugs.fixture_plug import FixturePlug @htf.plug(fixture=FixturePlug) @htf.measures( htf.Measurement('leakage_current').in_range(maximum=50).with_units(units.AMPERE), ) def measure_leakage(test, fixture): fixture.power_on_dut() fixture.close_test_relay() current_ua = read_current_meter() * 1e6 test.measurements.leakage_current = current_ua if not test.measurements.leakage_current.is_pass: test.logger.error('Leakage %.1f uA exceeds 50 uA limit', current_ua) return htf.PhaseResult.STOP # fixture.tearDown() called automatically by OpenHTF def main(): test = htf.Test(measure_leakage) with TofuPilot(test): test.execute(test_start=lambda: 'SN-004') ``` ## Failure Handling Strategy Comparison | Strategy | Use when | PhaseResult | Downstream phases | |---|---|---|---| | Continue on fail | Failures are independent, collect all data | (default) | Run | | Stop on first fail | Downstream tests are invalid after failure | `STOP` | Skipped | | Retry transient | Failure may be a fixture fluke | `REPEAT` | Run after retry | | Retry with limit | Same, but cap attempts | `REPEAT` + counter | Run or STOP | ## Logging Failures for Analytics TofuPilot captures every failed measurement automatically. To make failures actionable in analytics, add structured context using `test.logger`. ```python filename="tests/full_sequence.py" import openhtf as htf from openhtf.util import units from tofupilot.openhtf import TofuPilot @htf.measures( htf.Measurement('oscillator_freq_hz').in_range(7_990_000, 8_010_000).with_units(units.HERTZ), ) def check_oscillator(test): freq = measure_frequency(channel=2) test.measurements.oscillator_freq_hz = freq if not test.measurements.oscillator_freq_hz.is_pass: deviation_ppm = abs(freq - 8_000_000) / 8_000_000 * 1e6 test.logger.error( 'Oscillator %.0f Hz (%.1f ppm from 8 MHz)', freq, deviation_ppm, ) @htf.measures( htf.Measurement('adc_offset_mv').in_range(-5, 5), htf.Measurement('adc_gain_error_pct').within_percent(1.0, 0.1), ) def calibrate_adc(test): offset = measure_adc_offset() gain = measure_adc_gain() test.measurements.adc_offset_mv = offset * 1000 test.measurements.adc_gain_error_pct = gain def main(): test = htf.Test(check_oscillator, calibrate_adc, name='PCB-Rev-C') with TofuPilot(test): test.execute(test_start=lambda: 'SN-005') ``` Log messages appear in the TofuPilot run detail view alongside the failing measurement. Quantitative context (deviation in ppm, not just pass/fail) makes root cause analysis faster when reviewing failures across a batch. ### Run Parallel Tests on Multiple DUTs URL: https://www.tofupilot.com/guides/how-to-run-parallel-tests-on-multiple-duts-with-tofupilot Learn how to run independent OpenHTF test instances in parallel across multiple DUTs using Python multiprocessing, with per-DUT result logging to TofuPilot. Running tests on multiple devices under test simultaneously cuts cycle time and reduces the cost per unit. This guide shows how to structure a Python test station that runs independent OpenHTF test instances in parallel, each logging results to TofuPilot. ## Why Parallel DUT Testing Matters Sequential testing on a multi-fixture station leaves test capacity on the table. If your fixture holds four DUTs but your software tests one at a time, you're paying for hardware you're not using. | Strategy | 4-DUT Fixture, 30s Test | Throughput (units/hour) | |---|---|---| | Sequential | 4 x 30s = 120s per cycle | 30 | | Parallel (4 workers) | ~32s per cycle (overhead) | ~112 | Parallel testing is most valuable when: - Test time is dominated by I/O waits (serial reads, power supply settling, RF measurements) - Your fixture physically holds multiple DUTs simultaneously - Takt time is a bottleneck in manufacturing ## Architecture Overview Each DUT gets its own OpenHTF test instance running in a separate process. Results for each DUT are logged independently to TofuPilot. ```text filename="architecture.txt" Station Controller ├── DUT 0 → Process 0 → OpenHTF Test → TofuPilot (serial: SN-001) ├── DUT 1 → Process 1 → OpenHTF Test → TofuPilot (serial: SN-002) ├── DUT 2 → Process 2 → OpenHTF Test → TofuPilot (serial: SN-003) └── DUT 3 → Process 3 → OpenHTF Test → TofuPilot (serial: SN-004) ``` ## Choosing a Parallelism Model | Model | Best For | Shared State | GIL Impact | |---|---|---|---| | `multiprocessing` | CPU-bound or instrument-isolated DUTs | Requires IPC (Queue, Manager) | None | | `threading` | I/O-bound tests, shared instrument objects | Direct (with locks) | Applies | | `ProcessPoolExecutor` | Simple parallel dispatch, result collection | Queue / Manager | None | | `ThreadPoolExecutor` | Lightweight I/O-bound test phases | Direct (with locks) | Applies | For most hardware test stations, `multiprocessing` with one process per DUT is the safest choice. It isolates faults, avoids GIL contention, and keeps each DUT's logs and state fully independent. ## Step 1: Define Test Phases ```python filename="station/test_phases.py" import openhtf as htf from openhtf.plugs import BasePlug from openhtf.util import units class PowerSupplyPlug(BasePlug): """Mock power supply for a single DUT slot.""" def setUp(self): self._voltage = 5.01 self._current = 0.042 def measure_voltage(self) -> float: return self._voltage def measure_current(self) -> float: return self._current def tearDown(self): pass class SerialPortPlug(BasePlug): """Mock serial connection to a single DUT.""" def setUp(self): self._version = "2.4.1" def query(self, command: str) -> str: return self._version def tearDown(self): pass @htf.measures( htf.Measurement("supply_voltage") .in_range(minimum=4.8, maximum=5.2) .with_units(units.VOLT) ) @htf.plug(psu=PowerSupplyPlug) def measure_supply_voltage(test, psu): test.measurements.supply_voltage = psu.measure_voltage() @htf.measures( htf.Measurement("idle_current") .in_range(minimum=0.010, maximum=0.150) .with_units(units.AMPERE) ) @htf.plug(psu=PowerSupplyPlug) def measure_idle_current(test, psu): test.measurements.idle_current = psu.measure_current() @htf.measures( htf.Measurement("firmware_version").equals("2.4.1") ) @htf.plug(serial=SerialPortPlug) def check_firmware(test, serial): version = serial.query("VERSION?").strip() test.measurements.firmware_version = version ``` ## Step 2: Create the Worker Process Each worker creates its own OpenHTF `Test` object and runs it to completion. ```python filename="station/worker.py" import openhtf as htf from tofupilot.openhtf import TofuPilot from station.test_phases import ( measure_supply_voltage, measure_idle_current, check_firmware, ) def run_test_for_dut(slot_index: int, serial_number: str) -> str: """Entry point for a single DUT worker process.""" test = htf.Test( measure_supply_voltage, measure_idle_current, check_firmware, test_name=f"Production Test - Slot {slot_index}", ) with TofuPilot(test): test.execute(test_start=lambda: serial_number) return serial_number ``` ## Step 3: Launch All DUTs in Parallel The controller scans DUT serials from the fixture, then dispatches one process per slot. ```python filename="station/controller.py" import multiprocessing from concurrent.futures import ProcessPoolExecutor, as_completed from station.worker import run_test_for_dut def scan_serials() -> dict[int, str]: """Read serial numbers from the fixture.""" return { 0: "SN-2024-001", 1: "SN-2024-002", 2: "SN-2024-003", 3: "SN-2024-004", } def run_parallel_cycle(): serials = scan_serials() slots = list(serials.keys()) print(f"Starting parallel test cycle for {len(slots)} DUTs...") with ProcessPoolExecutor(max_workers=len(slots)) as executor: futures = { executor.submit(run_test_for_dut, slot, serials[slot]): slot for slot in slots } for future in as_completed(futures): slot = futures[future] try: sn = future.result() print(f"Slot {slot} ({sn}): complete") except Exception as exc: print(f"Slot {slot}: FAILED with {exc}") if __name__ == "__main__": multiprocessing.set_start_method("spawn") run_parallel_cycle() ``` ## Handling Partial Fixture Occupancy Not all fixture slots may be occupied on every cycle. Guard against empty slots before submitting futures. ```python filename="station/controller.py" def detect_present_duts() -> dict[int, str]: """Return only slots where a DUT is physically present.""" present = {} for slot in range(4): if fixture_sense_pin_active(slot): present[slot] = read_barcode(slot) return present ``` ## Comparison of Parallel Strategies | Approach | Fault Isolation | Shared Instrument Access | Recommended For | |---|---|---|---| | `ProcessPoolExecutor` | Full (separate processes) | Requires IPC or per-channel locks | Most production stations | | `ThreadPoolExecutor` | None (shared process) | Direct (locks required) | Lightweight I/O-bound tests | | `asyncio` with async VISA | None | Cooperative (no preemption) | Fully async instrument drivers | | One process per DUT (manual) | Full | Requires IPC | Custom fixture controllers | ## Common Pitfalls **Process start method on macOS and Windows.** Always call `multiprocessing.set_start_method("spawn")` before launching workers. The default `fork` method on macOS can deadlock when forking after VISA or serial port initialization. **Plug tearDown on test failure.** OpenHTF calls `tearDown()` on plugs even when a phase raises an exception. Make sure `tearDown` is idempotent and doesn't raise. **Serial number collision.** If two processes receive the same serial number, TofuPilot creates two runs for the same unit. Validate serial uniqueness in `scan_serials()` before dispatching. **Instrument timeout under load.** Shared instruments may time out when multiple channels are queried simultaneously. Tune per-channel lock timeouts and instrument read timeouts to match your slowest measurement. ### Build Your First Hardware Test URL: https://www.tofupilot.com/guides/how-to-build-your-first-hardware-test-with-python-and-tofupilot Build and run a hardware test in Python in 15 minutes using OpenHTF and TofuPilot, with measurements, limits, and automatic data logging. You can run a hardware test in Python in 15 minutes. This guide walks you through writing a functional test with OpenHTF and TofuPilot that measures voltages, checks limits, and logs results automatically. No LabVIEW, no TestStand, no license fees. ## Prerequisites - Python 3.9+ - pip (Python package manager) ## Step 1: Install the Dependencies ```bash filename="install.sh" pip install openhtf tofupilot ``` OpenHTF is Google's open-source hardware test framework. TofuPilot connects it to a cloud dashboard for analytics, traceability, and yield tracking. ## Step 2: Write Your First Test Phase A test phase is a Python function that takes measurements. OpenHTF handles the structure: you declare what you're measuring, set limits, and write the logic. ```python filename="first_test.py" import openhtf as htf from openhtf.util import units @htf.measures( htf.Measurement("voltage_3v3") .in_range(3.2, 3.4) .with_units(units.VOLT) .doc("3.3V rail voltage"), ) @htf.PhaseOptions(timeout_s=10) def test_voltage(test): test.measurements.voltage_3v3 = 3.31 # Replace with real instrument read ``` The `@htf.measures` decorator defines what this phase records. `.in_range(3.2, 3.4)` sets pass/fail limits. `.with_units()` adds units for analytics. The phase passes if the value falls within the range. ## Step 3: Add Multiple Measurements A single phase can measure several things. This is useful for testing all power rails in one step. ```python filename="multi_measurement.py" import openhtf as htf from openhtf.util import units @htf.measures( htf.Measurement("voltage_3v3") .in_range(3.2, 3.4) .with_units(units.VOLT) .doc("3.3V rail"), htf.Measurement("voltage_5v0") .in_range(4.8, 5.2) .with_units(units.VOLT) .doc("5.0V rail"), htf.Measurement("board_current") .in_range(0.05, 0.25) .with_units(units.AMPERE) .doc("Total board current draw"), ) def test_power_rails(test): test.measurements.voltage_3v3 = 3.31 test.measurements.voltage_5v0 = 5.02 test.measurements.board_current = 0.12 ``` ## Step 4: Use a Plug for Instrument Control Plugs manage instrument connections. OpenHTF calls `setUp()` before tests and `tearDown()` after, so your instruments connect and disconnect automatically. Plugs are injected into phases using the `@htf.plug()` decorator. ```python filename="plug_example.py" import openhtf as htf from openhtf.plugs import BasePlug from openhtf.util import units class MultimeterPlug(BasePlug): """Manage multimeter connection lifecycle.""" def setUp(self): # Replace with real instrument connection self._readings = iter([3.31, 5.02, 0.12]) def read_dc_voltage(self) -> float: return next(self._readings) def read_dc_current(self) -> float: return next(self._readings) def tearDown(self): pass # Replace with instrument disconnect @htf.measures( htf.Measurement("rail_3v3") .in_range(3.2, 3.4) .with_units(units.VOLT), ) @htf.plug(dmm=MultimeterPlug) def test_with_instrument(test, dmm): test.measurements.rail_3v3 = dmm.read_dc_voltage() ``` The `@htf.plug(dmm=MultimeterPlug)` decorator tells OpenHTF to create a `MultimeterPlug` instance and pass it as the `dmm` argument. Don't use type hints for plug injection (e.g., `dmm: MultimeterPlug`). The `@htf.plug()` decorator is required. ## Step 5: Add Different Measurement Types OpenHTF supports numeric ranges, exact matches, and boolean checks. ```python filename="measurement_types.py" import openhtf as htf from openhtf.util import units # Boolean: pass if True @htf.measures( htf.Measurement("led_on").equals(True).doc("Power LED is illuminated"), ) def test_led(test): test.measurements.led_on = True # String: pass if exact match @htf.measures( htf.Measurement("firmware_version").equals("2.1.0").doc("Expected firmware"), ) def test_firmware(test): test.measurements.firmware_version = "2.1.0" # Numeric range: pass if within limits @htf.measures( htf.Measurement("temperature") .in_range(20, 30) .with_units(units.DEGREE_CELSIUS) .doc("Board temperature"), ) def test_temperature(test): test.measurements.temperature = 24.5 ``` | Validator | Use Case | Example | |-----------|----------|---------| | `.in_range(low, high)` | Numeric within limits | Voltage, current, resistance | | `.equals(value)` | Exact match | Firmware version, boolean flags | | `.with_units(unit)` | Attach unit for analytics | `units.VOLT`, `units.AMPERE` | | `.doc(text)` | Description for reports | Shown in TofuPilot dashboard | ## Step 6: Assemble and Run the Test Connect the phases into a test, add TofuPilot for cloud logging, and run it. ```python filename="full_test.py" import openhtf as htf from openhtf.plugs import BasePlug from openhtf.util import units from tofupilot.openhtf import TofuPilot class MultimeterPlug(BasePlug): def setUp(self): self._readings = iter([3.31, 5.02, 0.12]) def read_dc_voltage(self) -> float: return next(self._readings) def read_dc_current(self) -> float: return next(self._readings) def tearDown(self): pass @htf.measures( htf.Measurement("voltage_3v3") .in_range(3.2, 3.4) .with_units(units.VOLT) .doc("3.3V rail"), htf.Measurement("voltage_5v0") .in_range(4.8, 5.2) .with_units(units.VOLT) .doc("5.0V rail"), htf.Measurement("board_current") .in_range(0.05, 0.25) .with_units(units.AMPERE) .doc("Board current draw"), ) @htf.plug(dmm=MultimeterPlug) def test_power(test, dmm): test.measurements.voltage_3v3 = dmm.read_dc_voltage() test.measurements.voltage_5v0 = dmm.read_dc_voltage() test.measurements.board_current = dmm.read_dc_current() @htf.measures( htf.Measurement("led_on").equals(True), ) def test_led(test): test.measurements.led_on = True @htf.measures( htf.Measurement("firmware_version").equals("2.1.0"), ) def test_firmware(test): test.measurements.firmware_version = "2.1.0" def main(): test = htf.Test( test_power, test_led, test_firmware, procedure_id="FCT-001", part_number="PCBA-100", ) with TofuPilot(test): test.execute(test_start=lambda: input("Scan serial number: ")) if __name__ == "__main__": main() ``` When you run this, OpenHTF prompts for a serial number, executes each phase in order, checks measurements against limits, and TofuPilot uploads the results. You get FPY, Cpk, and control charts in the dashboard with zero extra code. ## What Happens Behind the Scenes | Step | Who Does It | What Happens | |------|------------|--------------| | Serial number prompt | OpenHTF | Operator scans or types the DUT serial number | | Phase execution | OpenHTF | Runs `test_power`, `test_led`, `test_firmware` in order | | Measurement validation | OpenHTF | Checks each value against declared limits | | Result upload | TofuPilot | Sends measurements, limits, units, pass/fail to cloud | | Analytics | TofuPilot | FPY, Cpk, control charts, failure Pareto updated automatically | ### Structure a Production Test Script URL: https://www.tofupilot.com/guides/how-to-structure-a-python-test-script-for-production-with-tofupilot Learn how to organize an OpenHTF test script for production use, including phase ordering, plug management, configuration, and multi-SKU support with TofuPilot. A production test script runs hundreds of times a day. It needs to be reliable, maintainable, and easy for operators to use. This guide covers how to structure an OpenHTF test script that's ready for the production floor, not just your dev bench. ## Test Script Anatomy Every production test script has the same structure: ```text filename="structure.txt" imports plug classes (instrument drivers) phase functions (test steps) test assembly (connect everything) main (entry point) ``` Keep this order. It's what every test engineer on your team will expect. ## Step 1: Define Your Plugs Plugs manage instrument connections. One plug per instrument type. `setUp()` opens the connection. `tearDown()` closes it. OpenHTF handles the lifecycle automatically. ```python filename="production/plugs.py" import openhtf as htf from openhtf.plugs import BasePlug class PowerSupplyPlug(BasePlug): """Bench power supply control.""" def setUp(self): self.output_on = False # Replace: rm = pyvisa.ResourceManager("@py") # self.instr = rm.open_resource("TCPIP::192.168.1.101::INSTR") def enable(self): self.output_on = True # Replace: self.instr.write(":OUTP ON") def disable(self): self.output_on = False # Replace: self.instr.write(":OUTP OFF") def tearDown(self): self.disable() class DmmPlug(BasePlug): """Digital multimeter for voltage and current measurements.""" def setUp(self): self._readings = iter([3.31, 5.01, 0.12]) # Replace: rm = pyvisa.ResourceManager("@py") # self.instr = rm.open_resource("TCPIP::192.168.1.100::INSTR") def read_voltage(self) -> float: return next(self._readings) # Replace: return float(self.instr.query(":MEAS:VOLT:DC?")) def read_current(self) -> float: return next(self._readings) # Replace: return float(self.instr.query(":MEAS:CURR:DC?")) def tearDown(self): pass # Replace: self.instr.close() ``` **Rules for plugs:** - One plug per instrument type (not per instrument instance) - `tearDown()` must always leave the instrument safe (output off, connection closed) - Don't put measurement limits in plugs. Plugs read raw values. Phases apply limits. ## Step 2: Write Phase Functions Each phase tests one logical thing. Keep phases focused and independent. ```python filename="production/phases.py" import openhtf as htf from openhtf.util import units @htf.measures( htf.Measurement("rail_3v3") .in_range(3.2, 3.4) .with_units(units.VOLT) .doc("3.3V supply rail"), htf.Measurement("rail_5v0") .in_range(4.8, 5.2) .with_units(units.VOLT) .doc("5.0V supply rail"), ) @htf.plug(psu=PowerSupplyPlug, dmm=DmmPlug) def test_power_rails(test, psu, dmm): """Power the DUT and verify voltage rails.""" psu.enable() test.measurements.rail_3v3 = dmm.read_voltage() test.measurements.rail_5v0 = dmm.read_voltage() @htf.measures( htf.Measurement("idle_current") .in_range(0.05, 0.20) .with_units(units.AMPERE) .doc("Board idle current draw"), ) @htf.plug(dmm=DmmPlug) def test_current(test, dmm): """Measure idle current consumption.""" test.measurements.idle_current = dmm.read_current() ``` **Rules for phases:** - One `@htf.measures` block per phase. Declare everything the phase will measure. - Use `@htf.plug()` decorator, not type hints, for plug injection. - Phase names should describe what they test: `test_power_rails`, not `phase_1`. - Add `.doc()` to every measurement. It shows up in TofuPilot and reports. ## Step 3: Order Phases Correctly Phase order matters in production. Test the most critical thing first. If power rails fail, don't waste time testing communication. | Phase Order | Phase | Why This Order | |-------------|-------|---------------| | 1 | `test_power_rails` | If power fails, everything else will too | | 2 | `test_current` | High current = short = stop before damage | | 3 | `test_communication` | Verify firmware is alive before functional tests | | 4 | `test_functional` | Board-level behavior | | 5 | `test_calibration` | Fine-tuning (only if previous steps pass) | ## Step 4: Assemble the Test ```python filename="production/main.py" import openhtf as htf from tofupilot.openhtf import TofuPilot def main(): test = htf.Test( test_power_rails, test_current, procedure_id="FCT-001", part_number="PCBA-100", ) with TofuPilot(test): test.execute(test_start=lambda: input("Scan serial number: ")) if __name__ == "__main__": main() ``` `procedure_id` identifies the test type. `part_number` identifies the product. Both show up in TofuPilot for filtering and analytics. ## Step 5: Support Multiple SKUs When the same test station tests different products, use a factory function. ```python filename="production/multi_sku.py" import openhtf as htf from tofupilot.openhtf import TofuPilot SKU_CONFIG = { "PCBA-100": { "procedure_id": "FCT-001", "phases": [test_power_rails, test_current], }, "PCBA-200": { "procedure_id": "FCT-002", "phases": [test_power_rails, test_current], }, } def create_test(part_number: str) -> htf.Test: """Create a test configured for a specific SKU.""" config = SKU_CONFIG[part_number] return htf.Test( *config["phases"], procedure_id=config["procedure_id"], part_number=part_number, ) def main(): part_number = input("Scan part number: ").strip() if part_number not in SKU_CONFIG: print(f"Unknown part number: {part_number}") return test = create_test(part_number) with TofuPilot(test): test.execute(test_start=lambda: input("Scan serial number: ")) ``` ## File Organization For a single-product test station, one file is fine. For multi-product or complex tests, split into modules: ```text filename="project_structure.txt" fct/ ├── __init__.py ├── plugs/ │ ├── __init__.py │ ├── power_supply.py # PowerSupplyPlug │ ├── dmm.py # DmmPlug │ └── uart.py # UartPlug ├── phases/ │ ├── __init__.py │ ├── power.py # test_power_rails │ ├── current.py # test_current │ └── communication.py # test_communication ├── config.py # SKU configs, limits └── main.py # Test assembly and entry point ``` **When to split:** If your test file exceeds 300 lines, split it. If you have more than 5 plugs, split them into a plugs/ directory. If you test more than 3 SKUs, move config to its own file. ## Common Mistakes | Mistake | Problem | Fix | |---------|---------|-----| | Putting limits in plugs | Can't reuse plug for different products | Keep limits in `@htf.measures` | | Using type hints for plug injection | `dut: DutPlug` doesn't inject | Use `@htf.plug(dut=DutPlug)` decorator | | Not calling `tearDown()` in plugs | Instruments left in unknown state | Always implement `tearDown()` | | Testing everything in one phase | One failure masks others | Split into focused phases | | Hardcoding instrument addresses | Can't move to different station | Use config file or environment variables | | No timeout on phases | Hung test blocks the station | Add `@htf.PhaseOptions(timeout_s=30)` | ### Test Power Rails on a PCBA URL: https://www.tofupilot.com/guides/how-to-test-power-rails-on-a-pcba-with-python-and-tofupilot Learn how to test PCBA power rails with Python, including voltage regulator limits, current draw checks, sequencing, and automated logging via OpenHTF. Power rail testing validates that voltage regulators produce correct output before any downstream components are exercised. Run these checks first: a failed rail contaminates every subsequent measurement and risks damaging the device under test. ## Prerequisites - TofuPilot account and API key - OpenHTF and TofuPilot installed (`pip install openhtf tofupilot`) - Bench power supply or fixture with controllable power enable line - Digital multimeter or DAQ accessible from Python ## Common Power Rails on a PCBA | Rail | Typical Source | Nominal (V) | Common Tolerance | Load Range | |------|---------------|-------------|-----------------|------------| | 3.3V | LDO or buck (e.g. TPS62840) | 3.3 | +/-2% | 0-500 mA | | 5V | Boost or USB VBUS | 5.0 | +/-5% | 0-2 A | | 1.8V | LDO (e.g. TLV75518) | 1.8 | +/-2% | 0-300 mA | | 1.2V | Core LDO or buck | 1.2 | +/-3% | 0-1 A | | 12V | Boost or input direct | 12.0 | +/-5% | 0-3 A | | VBAT | Li-ion cell | 3.0-4.2 | varies | N/A | ## Setting Limits from Datasheets Pull limits directly from the voltage regulator datasheet. **TPS62840 (3.3V buck, Texas Instruments)** - Typical output accuracy: +/-1.5% at room temperature - Over temperature: +/-2.5% - Production test at room temp, use +/-2%: 3.234 V to 3.366 V **TLV75518 (1.8V LDO, Texas Instruments)** - Initial accuracy: +/-0.75% - Add +/-0.5% for resistor divider tolerance - Production test, use +/-1.5%: 1.773 V to 1.827 V Always add margin for your measurement system's accuracy. A 16-bit ADC with a 5 V reference introduces roughly +/-0.08 mV of quantization. ## Measurement Techniques | Measurement | Instrument | When to Use | |-------------|-----------|-------------| | DC voltage | DMM / ADC | Every unit | | No-load current | Current clamp or sense resistor | First article, incoming inspection | | Full-load current | Power supply readback | Functional test | | Ripple (peak-to-peak) | Oscilloscope or RMS ADC | Design validation, process escapes | ## Power Sequencing 1. Enable bench supply or fixture relay 2. Wait for the slowest regulator to settle (typically 1-5 ms) 3. Measure all rails 4. Execute functional tests 5. Disable rails in reverse order ## Complete OpenHTF Test with TofuPilot ```python filename="tests/power_rails_test.py" import time import openhtf as htf from openhtf.util import units from tofupilot.openhtf import TofuPilot class PowerFixturePlug(htf.plugs.BasePlug): """Controls fixture relay and reads voltages from a DAQ.""" def enable_power(self): # Assert fixture relay to apply supply voltage to DUT pass def disable_power(self): # De-assert fixture relay pass def read_voltage(self, channel: str) -> float: # Return DC voltage in volts on the named channel # Replace with your actual instrument driver voltages = { "rail_3v3": 3.312, "rail_1v8": 1.796, "rail_5v0": 4.985, } return voltages.get(channel, 0.0) def read_current_amps(self, channel: str) -> float: # Return current draw in amps via sense resistor currents = { "rail_3v3_current": 0.142, "rail_1v8_current": 0.085, "rail_5v0_current": 0.310, } return currents.get(channel, 0.0) def tearDown(self): self.disable_power() @htf.plug(fixture=PowerFixturePlug) def enable_and_settle(test, fixture): """Enable DUT power and wait for all regulators to settle.""" fixture.enable_power() time.sleep(0.010) # 10 ms covers TPS62840 and TLV75518 soft-start @htf.plug(fixture=PowerFixturePlug) @htf.measures( htf.Measurement("rail_3v3_voltage") .in_range(minimum=3.234, maximum=3.366) .with_units(units.VOLT) .doc("3.3V rail: TPS62840 +/-2% at room temp"), htf.Measurement("rail_3v3_current") .in_range(minimum=0.0, maximum=0.500) .with_units(units.AMPERE) .doc("3.3V rail current draw"), ) def check_3v3_rail(test, fixture): test.measurements.rail_3v3_voltage = fixture.read_voltage("rail_3v3") test.measurements.rail_3v3_current = fixture.read_current_amps("rail_3v3_current") @htf.plug(fixture=PowerFixturePlug) @htf.measures( htf.Measurement("rail_1v8_voltage") .in_range(minimum=1.773, maximum=1.827) .with_units(units.VOLT) .doc("1.8V rail: TLV75518 +/-1.5% including divider tolerance"), htf.Measurement("rail_1v8_current") .in_range(minimum=0.0, maximum=0.300) .with_units(units.AMPERE) .doc("1.8V rail current draw"), ) def check_1v8_rail(test, fixture): test.measurements.rail_1v8_voltage = fixture.read_voltage("rail_1v8") test.measurements.rail_1v8_current = fixture.read_current_amps("rail_1v8_current") @htf.plug(fixture=PowerFixturePlug) @htf.measures( htf.Measurement("rail_5v0_voltage") .in_range(minimum=4.750, maximum=5.250) .with_units(units.VOLT) .doc("5V boost rail +/-5%"), htf.Measurement("rail_5v0_current") .in_range(minimum=0.0, maximum=2.000) .with_units(units.AMPERE) .doc("5V rail current draw"), ) def check_5v0_rail(test, fixture): test.measurements.rail_5v0_voltage = fixture.read_voltage("rail_5v0") test.measurements.rail_5v0_current = fixture.read_current_amps("rail_5v0_current") @htf.plug(fixture=PowerFixturePlug) def disable_power_phase(test, fixture): """Disable DUT power after all measurements complete.""" fixture.disable_power() def main(): test = htf.Test( enable_and_settle, check_3v3_rail, check_1v8_rail, check_5v0_rail, disable_power_phase, test_name="Power Rails Test", ) with TofuPilot(test): test.execute(test_start=lambda: input("Enter serial number: ").strip()) if __name__ == "__main__": main() ``` ## Power Rail Failure Diagnostics | Symptom | Likely Cause | First Check | |---------|-------------|-------------| | Voltage reads 0 V | Regulator not enabled; blown input fuse | Check enable pin logic level; measure input voltage | | Voltage below minimum | Current limit reached; input undervoltage | Measure current draw; check input supply voltage drop | | Voltage above maximum | Feedback resistor open or wrong value | Measure feedback voltage divider; confirm BOM | | Voltage oscillates / noisy | Output cap missing or wrong type | Check ESR of output cap; inspect for tombstoned parts | | Correct voltage, excessive current | Short on downstream net | Disconnect loads one by one to isolate short | | Soft-start too slow | Large output cap; wrong soft-start resistor | Compare measured rise time to datasheet spec | ### Manage Multiple SKUs on a Test Station URL: https://www.tofupilot.com/guides/how-to-manage-multiple-skus-on-a-test-station-with-tofupilot Learn how to run multiple product SKUs on a single test station using config-driven phase selection, SKU-specific limits, and per-SKU yield tracking. One test station, multiple products: configure your OpenHTF test to detect the SKU at runtime, load SKU-specific limits and phases from a config file, and let TofuPilot track yield per part number automatically. ## Why Multi-SKU Stations Matter Running separate stations per SKU wastes floor space and fixtures. A single station that handles a product family reduces: - Fixture count (shared bed-of-nails or probe card) - Operator training (one workflow, not five) - Maintenance surface (one station to calibrate) ## SKU Detection Methods | Method | When to Use | Reliability | |--------|-------------|-------------| | Barcode scan | Operator-scanned label, no electrical contact needed | High | | Resistor ID | PCB has a resistor divider encoding SKU | Medium | | I2C EEPROM | PCB stores part number in onboard memory | High | ### Barcode Scan ```python filename="station/plugs/barcode_plug.py" import openhtf as htf class BarcodePlug(htf.plugs.BasePlug): """Reads SKU from operator barcode scan.""" def setUp(self): pass def scan(self) -> str: raw = input("Scan barcode: ").strip() if not raw: raise ValueError("Empty barcode scan") return raw def tearDown(self): pass ``` ### Resistor ID ```python filename="station/plugs/resistor_id_plug.py" import openhtf as htf _VOLTAGE_MAP = { (0.0, 0.4): "SKU-A", (0.4, 0.8): "SKU-B", (0.8, 1.2): "SKU-C", (1.2, 1.6): "SKU-D", } class ResistorIdPlug(htf.plugs.BasePlug): """Identifies SKU from resistor divider voltage.""" def setUp(self): pass def read_sku(self, adc_voltage: float) -> str: for (low, high), sku in _VOLTAGE_MAP.items(): if low <= adc_voltage < high: return sku raise ValueError(f"Unrecognised voltage: {adc_voltage:.3f} V") def tearDown(self): pass ``` ## Configuration-Driven Test Selection Store SKU definitions in a YAML file. This separates limits from test logic. ```yaml filename="station/config/skus.yaml" SKU-A: part_number: "PCB-001-A" description: "Standard 5 V variant" phases: - power_on - voltage_check - current_check limits: supply_voltage: min: 4.85 max: 5.15 supply_current: min: 0.080 max: 0.120 SKU-B: part_number: "PCB-001-B" description: "Low-power 3.3 V variant" phases: - power_on - voltage_check - current_check - sleep_current_check limits: supply_voltage: min: 3.2 max: 3.4 supply_current: min: 0.030 max: 0.060 sleep_current: min: 0 max: 0.000050 ``` Load the config at station startup: ```python filename="station/config/loader.py" from pathlib import Path import yaml def load_sku_config(path: str = "station/config/skus.yaml") -> dict: config_path = Path(path) if not config_path.exists(): raise FileNotFoundError(f"SKU config not found: {config_path}") with config_path.open() as f: return yaml.safe_load(f) ``` ## SKU-Specific Measurement Limits with OpenHTF Use a phase factory to bake SKU-specific limits into OpenHTF measurements: ```python filename="station/phases/factory.py" import openhtf as htf from openhtf.util import units def make_voltage_phase(min_v: float, max_v: float): """Returns a voltage phase with baked-in limits.""" @htf.measures( htf.Measurement("supply_voltage") .in_range(minimum=min_v, maximum=max_v) .with_units(units.VOLT), ) def voltage_check(test): test.measurements.supply_voltage = read_voltage() return voltage_check def make_current_phase(min_a: float, max_a: float): """Returns a current phase with baked-in limits.""" @htf.measures( htf.Measurement("supply_current") .in_range(minimum=min_a, maximum=max_a) .with_units(units.AMPERE), ) def current_check(test): test.measurements.supply_current = read_current() return current_check ``` ## Full Working Example ```python filename="station/main.py" import openhtf as htf from tofupilot.openhtf import TofuPilot from station.config.loader import load_sku_config from station.phases.factory import make_voltage_phase, make_current_phase SKU_CONFIG = load_sku_config() def power_on(test): """Powers on the DUT and waits for it to stabilise.""" import time time.sleep(0.5) def run_station(): # 1. Detect SKU sku_id = input("Scan barcode: ").strip() if sku_id not in SKU_CONFIG: raise ValueError(f"Unknown SKU '{sku_id}'. Available: {list(SKU_CONFIG)}") sku = SKU_CONFIG[sku_id] limits = sku["limits"] part_number = sku["part_number"] # 2. Compose phases from config phases = [power_on] if "supply_voltage" in limits: lim = limits["supply_voltage"] phases.append(make_voltage_phase(lim["min"], lim["max"])) if "supply_current" in limits: lim = limits["supply_current"] phases.append(make_current_phase(lim["min"], lim["max"])) # 3. Build and run test test = htf.Test(*phases, test_name=f"Station ({sku_id})") with TofuPilot(test): test.execute(test_start=lambda: input("Scan serial number: ")) if __name__ == "__main__": run_station() ``` ## Tracking Per-SKU Yield in TofuPilot TofuPilot groups yield statistics by part number. Each SKU gets its own yield chart without extra configuration. | Part Number | SKU | Runs | Pass Rate | |-------------|-----|------|-----------| | PCB-001-A | SKU-A | 240 | 97.5% | | PCB-001-B | SKU-B | 180 | 94.4% | Filter by part number in the TofuPilot dashboard to compare SKU trends side by side. ### How to Write a Hardware Test Plan URL: https://www.tofupilot.com/guides/how-to-write-a-hardware-test-plan-with-tofupilot A hardware test plan defines what to test, when, and how. Learn how to structure a test plan and implement it in Python with TofuPilot. # How to Write a Hardware Test Plan with TofuPilot A hardware test plan defines what gets tested, at which stage, with what equipment, and against which criteria. Without one, test coverage depends on whoever wrote the last script. This guide walks through structuring a test plan from EVT through production and implementing it in Python with TofuPilot. ## What a Test Plan Covers | Section | Content | |---------|---------| | Product overview | What the product does, key specifications | | Test stages | Which tests run at EVT, DVT, PVT, production | | Test procedures | Step-by-step instructions per test | | Equipment list | Instruments, fixtures, software | | Pass/fail criteria | Measurement limits per test | | Data requirements | What gets recorded, where, retention period | | Roles | Who writes tests, who runs them, who reviews results | The test plan is a living document. It starts rough in EVT and gets refined through each build stage. ## Step 1: Map Requirements to Test Stages Start with the product requirements and decide when each one gets verified. | Requirement | EVT | DVT | PVT | Production | |-------------|-----|-----|-----|------------| | Output voltage within 5% | X | X | X | X | | Survives 1m drop | | X | | | | Operates -20C to 60C | | X | X | | | Firmware boots in under 2s | X | X | X | X | | Leakage current below 500uA | | X | | X | | FPY above 95% | | | X | X | Not every requirement needs testing at every stage. EVT focuses on basic functionality. DVT covers environmental and stress. PVT validates the manufacturing process. Production tests run on every unit. ## Step 2: Define Test Procedures Each test procedure becomes an OpenHTF test script. Define the measurements and limits directly in code so the test plan and the implementation stay in sync. ```python filename="production_test.py" import openhtf as htf from openhtf.util import units @htf.measures( htf.Measurement("output_voltage_V") .in_range(minimum=4.75, maximum=5.25) .with_units(units.VOLT), htf.Measurement("boot_time_ms") .in_range(maximum=2000) .with_units(units.MILLISECOND), htf.Measurement("leakage_current_uA") .in_range(maximum=500) .with_units(units.MICROAMPERE), ) def phase_production_checks(test): """Production test: voltage, boot time, safety.""" test.measurements.output_voltage_V = 5.02 test.measurements.boot_time_ms = 1350 test.measurements.leakage_current_uA = 85.0 ``` ## Step 3: Specify Equipment and Fixtures Document every instrument in the test plan. This prevents "it works on my bench" problems. | Instrument | Model | Purpose | Calibration | |-----------|-------|---------|-------------| | DMM | Keysight 34461A | Voltage and current | Annual | | Power supply | Rigol DP832 | DUT power | Annual | | Fixture | Custom PCB jig v2.1 | Pogo pin contact | Per shift check | | PC | Any, Python 3.10+ | Test execution | N/A | ## Step 4: Set Initial Limits Start with limits from the datasheet or design spec. Tighten them after DVT data shows the actual distribution. ```python filename="limit_evolution.py" import openhtf as htf from openhtf.util import units # EVT: wide limits, learning the design evT_voltage = htf.Measurement("voltage_V").in_range( minimum=4.5, maximum=5.5 ).with_units(units.VOLT) # DVT: tightened based on EVT data dvt_voltage = htf.Measurement("voltage_V").in_range( minimum=4.7, maximum=5.3, marginal_minimum=4.75, marginal_maximum=5.25, ).with_units(units.VOLT) # Production: final limits with margins prod_voltage = htf.Measurement("voltage_V").in_range( minimum=4.75, maximum=5.25, marginal_minimum=4.8, marginal_maximum=5.2, ).with_units(units.VOLT) ``` ## Step 5: Connect to TofuPilot Log every test run to TofuPilot from the start. EVT data informs DVT limits. DVT data informs production limits. This only works if the data is captured consistently. ```python filename="production_test.py" from tofupilot.openhtf import TofuPilot test = htf.Test(phase_production_checks) with TofuPilot(test): test.execute(test_start=lambda: input("Scan serial: ")) ``` TofuPilot tracks results across all test stages. Open the Analytics tab to see measurement distributions, yield trends, and failure patterns. This data feeds back into the test plan as you refine limits and add or remove test steps. ## Common Mistakes | Mistake | Fix | |---------|-----| | Testing everything at every stage | Map requirements to specific stages | | Limits from the datasheet only | Refine from actual production data | | Test plan in a Word doc, code elsewhere | Define limits in code, keep them in sync | | No calibration tracking | Log instrument info with each run | | Skipping EVT data collection | Start logging from the first prototype | A test plan is not a one-time document. Review it after every build stage and update limits, procedures, and equipment based on what the data shows. ### How to Control a Keysight DMM with TofuPilot URL: https://www.tofupilot.com/guides/how-to-control-a-keysight-multimeter-with-python-and-tofupilot Connect a Keysight 34461A or 34465A DMM to Python using PyVISA, measure voltage, current, and resistance, and log results to TofuPilot via an OpenHTF plug. Connect a Keysight 34461A or 34465A DMM to Python using PyVISA, send SCPI commands to measure DC voltage, AC voltage, resistance, and current, then log results automatically to TofuPilot via an OpenHTF plug. ## Prerequisites - Keysight 34461A or 34465A DMM (or compatible 344xxA series) - Python 3.8+ - OpenHTF and TofuPilot installed - Keysight IO Libraries Suite (for USB) or network access (for Ethernet) ```bash filename="terminal" pip install pyvisa pyvisa-py tofupilot openhtf ``` ## Step 1: Connect to the Instrument Keysight DMMs support two primary interfaces: USB-TMC and Ethernet (LXI). | Interface | VISA Resource String | Use Case | |-----------|---------------------|----------| | USB-TMC | `USB0::0x0957::0x1A07::MY12345678::INSTR` | Single bench, direct PC connection | | Ethernet (VXI-11) | `TCPIP0::192.168.1.100::inst0::INSTR` | Networked test systems | | Ethernet (HiSLIP) | `TCPIP0::192.168.1.100::hislip0::INSTR` | Higher throughput, recommended for 34465A | Find your instrument's resource string: ```python filename="find_instruments.py" import pyvisa rm = pyvisa.ResourceManager() print(rm.list_resources()) ``` Open a connection and verify identity: ```python filename="connect.py" import pyvisa rm = pyvisa.ResourceManager() dmm = rm.open_resource("USB0::0x0957::0x1A07::MY12345678::INSTR") dmm.timeout = 5000 # ms print(dmm.query("*IDN?")) # Keysight Technologies,34461A,MY12345678,A.02.14-02.40-02.14-00.49-01-01 ``` ## Step 2: Configure SCPI Measurements Always reset to a known state before configuring. ```python filename="scpi_basics.py" dmm.write("*RST") dmm.write("*CLS") print(dmm.query("SYST:ERR?")) # +0,"No error" ``` ### DC Voltage ```python filename="measure_dc_voltage.py" # Auto-range DC voltage dmm.write("CONF:VOLT:DC AUTO,DEF") reading = float(dmm.query("READ?")) print(f"DC Voltage: {reading:.6f} V") # Manual range: 10 V range dmm.write("CONF:VOLT:DC 10,DEF") reading = float(dmm.query("READ?")) ``` ### AC Voltage ```python filename="measure_ac_voltage.py" dmm.write("CONF:VOLT:AC AUTO,DEF") reading = float(dmm.query("READ?")) print(f"AC Voltage (RMS): {reading:.6f} V") ``` ### Resistance (2-wire and 4-wire) ```python filename="measure_resistance.py" # 2-wire resistance, auto-range dmm.write("CONF:RES AUTO,DEF") reading = float(dmm.query("READ?")) print(f"2-wire resistance: {reading:.4f} Ohm") # 4-wire resistance (eliminates lead resistance) dmm.write("CONF:FRES AUTO,DEF") reading = float(dmm.query("READ?")) print(f"4-wire resistance: {reading:.4f} Ohm") ``` ### DC Current ```python filename="measure_current.py" dmm.write("CONF:CURR:DC AUTO,DEF") reading = float(dmm.query("READ?")) print(f"DC Current: {reading:.6f} A") ``` ### SCPI Command Reference | Measurement | CONF Command | Range Values | |-------------|-------------|-------| | DC Voltage | `CONF:VOLT:DC {range},DEF` | 0.1, 1, 10, 100, 1000 V | | AC Voltage | `CONF:VOLT:AC {range},DEF` | 0.1, 1, 10, 100, 750 V | | 2-wire Resistance | `CONF:RES {range},DEF` | 100, 1k, 10k, 100k, 1M, 10M, 100M Ohm | | 4-wire Resistance | `CONF:FRES {range},DEF` | Use for less than 100 Ohm | | DC Current | `CONF:CURR:DC {range},DEF` | 0.0001, 0.001, 0.01, 0.1, 1, 3 A | | AC Current | `CONF:CURR:AC {range},DEF` | 0.001, 0.01, 0.1, 1, 3 A | ## Step 3: Tune Speed vs. Accuracy with NPLC NPLC (Number of Power Line Cycles) controls integration time. Higher NPLC averages more noise but takes longer. ```python filename="nplc_config.py" dmm.write("CONF:VOLT:DC 10,DEF") dmm.write("VOLT:DC:NPLC 1") # balanced (default) dmm.write("VOLT:DC:NPLC 10") # high accuracy, slower dmm.write("VOLT:DC:NPLC 0.02") # fast, more noise ``` | NPLC | Integration Time (60 Hz) | Readings/sec | Use Case | |------|--------------------------|--------------|----------| | 0.02 | 333 us | ~50 | Fast production screening | | 0.2 | 3.3 ms | ~15 | Good speed/noise balance | | 1 | 16.7 ms | ~5 | Standard accuracy | | 10 | 167 ms | ~0.6 | High accuracy measurements | | 100 | 1.67 s | ~0.1 | Calibration-grade | Manual range eliminates the auto-range delay (~20-50 ms) and is recommended for production test throughput. ## Step 4: Build a Keysight DMM OpenHTF Plug ```python filename="keysight_dmm_plug.py" import pyvisa import openhtf as htf class KeysightDMM(htf.plugs.BasePlug): """OpenHTF plug for Keysight 344xxA series DMMs.""" RESOURCE = "USB0::0x0957::0x1A07::MY12345678::INSTR" NPLC = 1.0 def setUp(self): self._rm = pyvisa.ResourceManager() self._dmm = self._rm.open_resource(self.RESOURCE) self._dmm.timeout = 5000 self._dmm.write("*RST") self._dmm.write("*CLS") def tearDown(self): if self._dmm: self._dmm.close() self._rm.close() def measure_dc_voltage(self, range_v: float = None) -> float: range_str = str(range_v) if range_v else "AUTO" self._dmm.write(f"CONF:VOLT:DC {range_str},DEF") self._dmm.write(f"VOLT:DC:NPLC {self.NPLC}") return float(self._dmm.query("READ?")) def measure_ac_voltage(self, range_v: float = None) -> float: range_str = str(range_v) if range_v else "AUTO" self._dmm.write(f"CONF:VOLT:AC {range_str},DEF") return float(self._dmm.query("READ?")) def measure_resistance(self, four_wire: bool = False, range_ohm: float = None) -> float: cmd = "FRES" if four_wire else "RES" range_str = str(range_ohm) if range_ohm else "AUTO" self._dmm.write(f"CONF:{cmd} {range_str},DEF") self._dmm.write(f"{cmd}:NPLC {self.NPLC}") return float(self._dmm.query("READ?")) def measure_dc_current(self, range_a: float = None) -> float: range_str = str(range_a) if range_a else "AUTO" self._dmm.write(f"CONF:CURR:DC {range_str},DEF") self._dmm.write(f"CURR:DC:NPLC {self.NPLC}") return float(self._dmm.query("READ?")) ``` ## Step 5: Write Production Tests with TofuPilot ```python filename="test_power_supply_board.py" import openhtf as htf from openhtf.util import units from tofupilot.openhtf import TofuPilot from keysight_dmm_plug import KeysightDMM @htf.plug(dmm=KeysightDMM) @htf.measures( htf.Measurement("output_voltage_3v3") .in_range(minimum=3.267, maximum=3.333) .with_units(units.VOLT), htf.Measurement("output_voltage_5v0") .in_range(minimum=4.900, maximum=5.100) .with_units(units.VOLT), ) def test_output_voltages(test, dmm): """Verify regulated output voltages are within tolerance.""" test.measurements.output_voltage_3v3 = dmm.measure_dc_voltage(range_v=10) test.measurements.output_voltage_5v0 = dmm.measure_dc_voltage(range_v=10) @htf.plug(dmm=KeysightDMM) @htf.measures( htf.Measurement("sense_resistor_value") .in_range(minimum=0.099, maximum=0.101) .with_units(units.OHM), ) def test_sense_resistor(test, dmm): """Verify current sense resistor with 4-wire measurement.""" r = dmm.measure_resistance(four_wire=True, range_ohm=100) test.measurements.sense_resistor_value = r @htf.plug(dmm=KeysightDMM) @htf.measures( htf.Measurement("quiescent_current") .in_range(maximum=0.005) .with_units(units.AMPERE), ) def test_quiescent_current(test, dmm): """Verify quiescent current does not exceed 5 mA.""" test.measurements.quiescent_current = dmm.measure_dc_current(range_a=0.01) @htf.plug(dmm=KeysightDMM) @htf.measures( htf.Measurement("ac_ripple") .in_range(maximum=0.050) .with_units(units.VOLT), ) def test_output_ripple(test, dmm): """Verify AC ripple on the 5 V output.""" test.measurements.ac_ripple = dmm.measure_ac_voltage(range_v=1) def main(): test = htf.Test( test_output_voltages, test_sense_resistor, test_quiescent_current, test_output_ripple, test_name="Power Supply Board Functional Test", ) with TofuPilot(test): test.execute(test_start=lambda: input("Enter DUT serial number: ").strip()) if __name__ == "__main__": main() ``` ## Step 6: Fast Multi-Measurement Mode For high-throughput lines, use `SAMP:COUN` for burst sampling. ```python filename="fast_sampling.py" import pyvisa import numpy as np rm = pyvisa.ResourceManager() dmm = rm.open_resource("USB0::0x0957::0x1A07::MY12345678::INSTR") dmm.timeout = 10000 dmm.write("*RST") dmm.write("CONF:VOLT:DC 10,DEF") dmm.write("VOLT:DC:NPLC 0.2") dmm.write("SAMP:COUN 10") dmm.write("TRIG:SOUR IMM") dmm.write("TRIG:DEL 0") dmm.write("INIT") raw = dmm.query("FETC?") readings = [float(x) for x in raw.split(",")] print(f"Mean: {np.mean(readings):.6f} V") print(f"Std: {np.std(readings):.6f} V") ``` ## Troubleshooting | Symptom | Likely Cause | Fix | |---------|-------------|-----| | `VI_ERROR_RSRC_NFOUND` | Wrong resource string or driver not installed | Run `rm.list_resources()`, install Keysight IO Libraries | | `+1,"Hardware error"` | Overload on input terminals | Check probe connections, reduce input signal | | Reading returns `+9.9E+37` | Signal exceeds selected range | Use auto-range or increase manual range | | Readings drift over 30 seconds | DMM not warmed up | Allow 30-minute warm-up for less than 10 ppm accuracy | | USB device not found on Linux | Missing udev rules | Add `SUBSYSTEM=="usb", ATTRS{idVendor}=="0957", MODE="0666"` | | HiSLIP connection refused | HiSLIP not enabled | Enable via front panel: Utilities > I/O Config > LAN > HiSLIP | | `NPLC` command returns error | Wrong function prefix | NPLC is per-function: `VOLT:DC:NPLC`, `RES:NPLC` | | Slow throughput over LAN | VXI-11 overhead | Switch to HiSLIP for 10x faster transactions | | 4-wire reading matches 2-wire | Wrong terminals | Confirm SENSE HI/LO connected to separate terminals | ### Control Multiple Instruments in a Test URL: https://www.tofupilot.com/guides/how-to-control-multiple-instruments-in-a-test-with-tofupilot Learn how to orchestrate multiple instruments (PSU, DMM, serial port) in a single OpenHTF test using plugs, with automatic lifecycle management and logging. Functional testing rarely involves a single instrument. A real board test powers the DUT through a PSU, measures voltage and current with a DMM, and validates firmware communication over a serial port. OpenHTF handles this through the plug system: each instrument gets its own plug, phases declare which plugs they need, and OpenHTF manages lifecycle and injection. ## Prerequisites - Python 3.8+ - TofuPilot account and API key - OpenHTF and TofuPilot installed (`pip install openhtf tofupilot`) - Instrument drivers (`pyvisa`, `pyserial`) ## Why Multi-Instrument Tests Benefit from Plugs | Approach | Teardown | Sharing Across Phases | Testability | |---|---|---|---| | Global variables | Manual, error-prone | Implicit | Hard to mock | | Pass instruments as args | Manual | Explicit but verbose | Moderate | | OpenHTF plugs | Automatic via `tearDown` | Declarative injection | Easy to mock | Each plug is a class that owns one instrument connection. OpenHTF instantiates it once per test run, injects it into every phase that requests it, and calls `tearDown` when the test completes. ## Step 1: Define One Plug Per Instrument ```python filename="plugs/psu_plug.py" import openhtf as htf import pyvisa class PSUPlug(htf.plugs.BasePlug): """Controls a SCPI-compliant bench power supply.""" RESOURCE = "USB0::0x1AB1::0x0E11::DP8B213601234::INSTR" def setUp(self): rm = pyvisa.ResourceManager() self._instrument = rm.open_resource(self.RESOURCE) self._instrument.timeout = 5000 def enable_output(self, channel: int, voltage: float, current_limit: float): self._instrument.write(f"INST CH{channel}") self._instrument.write(f"VOLT {voltage}") self._instrument.write(f"CURR {current_limit}") self._instrument.write("OUTP ON") def disable_output(self, channel: int): self._instrument.write(f"INST CH{channel}") self._instrument.write("OUTP OFF") def measure_voltage(self, channel: int) -> float: self._instrument.write(f"INST CH{channel}") return float(self._instrument.query("MEAS:VOLT?")) def measure_current(self, channel: int) -> float: self._instrument.write(f"INST CH{channel}") return float(self._instrument.query("MEAS:CURR?")) def tearDown(self): self._instrument.write("OUTP:ALL OFF") self._instrument.close() ``` ```python filename="plugs/dmm_plug.py" import openhtf as htf import pyvisa class DMMPlug(htf.plugs.BasePlug): """Controls a 6.5-digit DMM over GPIB.""" RESOURCE = "GPIB0::22::INSTR" def setUp(self): rm = pyvisa.ResourceManager() self._instrument = rm.open_resource(self.RESOURCE) self._instrument.timeout = 10000 self._instrument.write("*RST") def measure_dc_voltage(self) -> float: self._instrument.write("CONF:VOLT:DC") return float(self._instrument.query("READ?")) def measure_resistance(self) -> float: self._instrument.write("CONF:RES") return float(self._instrument.query("READ?")) def tearDown(self): self._instrument.write("*RST") self._instrument.close() ``` ```python filename="plugs/serial_plug.py" import openhtf as htf import serial import time class SerialPlug(htf.plugs.BasePlug): """Communicates with DUT firmware over UART.""" PORT = "/dev/ttyUSB0" BAUDRATE = 115200 def setUp(self): self._port = serial.Serial( port=self.PORT, baudrate=self.BAUDRATE, timeout=2.0 ) time.sleep(0.1) def send_command(self, command: str) -> str: self._port.write(f"{command}\r\n".encode()) return self._port.readline().decode().strip() def get_firmware_version(self) -> str: return self.send_command("VERSION?") def run_self_test(self) -> bool: return self.send_command("SELFTEST") == "PASS" def tearDown(self): self._port.close() ``` ## Step 2: Write Phases That Declare Their Plugs Each phase receives the plugs it needs through the `@htf.plug` decorator. Phases only declare the instruments they actually use. ```python filename="phases/power_phases.py" import time import openhtf as htf from openhtf.util import units from plugs.psu_plug import PSUPlug from plugs.dmm_plug import DMMPlug @htf.plug(psu=PSUPlug) @htf.measures( htf.Measurement("supply_voltage_v12") .in_range(minimum=11.8, maximum=12.2) .with_units(units.VOLT), htf.Measurement("supply_current") .in_range(minimum=0.05, maximum=2.0) .with_units(units.AMPERE), ) def power_on_dut(test, psu): """Enable 12V rail and verify supply is within tolerance.""" psu.enable_output(channel=1, voltage=12.0, current_limit=2.5) time.sleep(0.5) test.measurements.supply_voltage_v12 = psu.measure_voltage(channel=1) test.measurements.supply_current = psu.measure_current(channel=1) @htf.plug(psu=PSUPlug, dmm=DMMPlug) @htf.measures( htf.Measurement("output_rail_3v3") .in_range(minimum=3.267, maximum=3.333) .with_units(units.VOLT), htf.Measurement("load_resistance") .in_range(minimum=95.0, maximum=105.0) .with_units(units.OHM), ) def verify_power_rails(test, psu, dmm): """Cross-check onboard 3.3V rail with external DMM measurement.""" test.measurements.output_rail_3v3 = dmm.measure_dc_voltage() test.measurements.load_resistance = dmm.measure_resistance() ``` ```python filename="phases/firmware_phases.py" import openhtf as htf from plugs.serial_plug import SerialPlug EXPECTED_FW_VERSION = "2.4.1" @htf.plug(serial=SerialPlug) @htf.measures( htf.Measurement("firmware_version").equals(EXPECTED_FW_VERSION), htf.Measurement("self_test_passed").equals(True), ) def verify_firmware(test, serial): """Check firmware version and run onboard self-test.""" test.measurements.firmware_version = serial.get_firmware_version() test.measurements.self_test_passed = serial.run_self_test() @htf.plug(serial=SerialPlug) @htf.measures( htf.Measurement("adc_reading_mv") .in_range(minimum=1180, maximum=1220), ) def verify_adc_calibration(test, serial): """Command DUT to report its ADC reading of the 1.2V reference.""" response = serial.send_command("ADC:REF?") test.measurements.adc_reading_mv = float(response) ``` ## Step 3: Sequence Phases in the Correct Order Instrument sequencing matters. Power the DUT before communicating with its firmware. ```text filename="phase_order.txt" power_on_dut → verify_power_rails → verify_firmware → verify_adc_calibration ↑ PSU only ↑ PSU + DMM ↑ Serial only ↑ Serial only ``` OpenHTF instantiates each plug once and reuses the same instance across all phases that request it. When `power_on_dut` and `verify_power_rails` both declare `psu=PSUPlug`, they receive the same `PSUPlug` instance. ## Step 4: Assemble and Run the Full Test ```python filename="test_board_functional.py" import openhtf as htf from tofupilot.openhtf import TofuPilot from phases.power_phases import power_on_dut, verify_power_rails from phases.firmware_phases import verify_firmware, verify_adc_calibration def main(): test = htf.Test( power_on_dut, verify_power_rails, verify_firmware, verify_adc_calibration, test_name="Board Functional Test", ) with TofuPilot(test): test.execute(test_start=lambda: input("Enter DUT serial number: ")) if __name__ == "__main__": main() ``` When executed, OpenHTF: 1. Instantiates `PSUPlug`, `DMMPlug`, and `SerialPlug` before the first phase 2. Runs phases in order, injecting the shared instances 3. Calls `tearDown` on all three plugs after the last phase, regardless of pass/fail 4. TofuPilot uploads the full run with all measurements ## Teardown Order and Safety Plugs tear down in reverse instantiation order by default. Design your teardown to be safe regardless of order: always disable outputs in `tearDown`, never assume another plug is still active. ```python filename="plugs/psu_plug.py" def tearDown(self): try: self._instrument.write("OUTP:ALL OFF") self._instrument.close() except Exception: pass # instrument may already be disconnected ``` ## Sharing Instruments Across Phase Files Because OpenHTF creates one plug instance per class per test run, you can split phases across multiple files and still share the same instrument connection. ```python filename="test_board_functional.py" # Both files import the same PSUPlug class. # OpenHTF injects the same instance into both phases. from phases.power_phases import power_on_dut, verify_power_rails from phases.stress_phases import run_load_step # also uses PSUPlug ``` This works without any extra configuration. ## Comparison: Single vs. Multi-Instrument Test Structure | Concern | Single instrument | Multiple instruments | |---|---|---| | Plug count | 1 | 1 per instrument | | Phase declarations | `@htf.plug(inst=MyPlug)` | Each phase declares only what it needs | | Teardown | Automatic | Automatic, one `tearDown` per plug | | Sequencing | N/A | Power on before firmware, disable in reverse | | TofuPilot upload | All measurements from one plug | All measurements from all plugs, unified | ### What Is Acceptance Testing for Hardware URL: https://www.tofupilot.com/guides/what-is-acceptance-testing-with-tofupilot Acceptance testing verifies a product meets agreed-upon criteria before delivery. Learn how to structure acceptance tests in Python and track results with. # What Is Acceptance Testing with TofuPilot Acceptance testing verifies that a product meets the criteria agreed upon between supplier and customer before delivery. It's the contractual gate: if the unit passes, the customer accepts it. This guide covers what acceptance testing involves for hardware, how to build acceptance test procedures in Python, and how to generate auditable records with TofuPilot. ## Types of Acceptance Testing | Type | Who Runs It | Where | Purpose | |------|------------|-------|---------| | FAT (Factory Acceptance Test) | Supplier | Supplier's facility | Prove it works before shipping | | SAT (Site Acceptance Test) | Customer or supplier | Customer's facility | Prove it works after installation | | ATP (Acceptance Test Procedure) | Either | Either | The documented test procedure itself | FAT and SAT often run the same test procedure but in different environments. The ATP defines the exact steps, measurements, and pass/fail criteria both parties agreed to. ## What Makes Acceptance Testing Different Acceptance testing is not exploratory. Every step is pre-defined, and the customer typically reviews and approves the ATP before testing begins. | Aspect | Acceptance Test | Production Test | |--------|----------------|-----------------| | Scope | Per contract requirements | Per design spec | | Documentation | Formal report required | Internal records | | Witness | Customer may be present | Internal only | | Criteria | Contractually agreed limits | Engineering limits | | Frequency | Per batch or per unit | Every unit | The key difference: acceptance test limits come from the contract, not from engineering margins. They're often tighter than production limits because the customer specifies what they need, not what the design can do. ## Prerequisites - Python 3.10+ - OpenHTF installed (`pip install openhtf`) - TofuPilot Python SDK installed (`pip install tofupilot`) ## Step 1: Define the Acceptance Criteria Map each ATP requirement to an OpenHTF measurement with the contractual limits. ```python filename="acceptance_test.py" import openhtf as htf from openhtf.util import units @htf.measures( htf.Measurement("output_power_W") .in_range(minimum=95, maximum=105) .with_units(units.WATT), htf.Measurement("efficiency_percent") .in_range(minimum=92) .with_units(units.PERCENT), ) def phase_power_output(test): """ATP Section 4.1: Output power and efficiency.""" test.measurements.output_power_W = 100.3 test.measurements.efficiency_percent = 94.7 @htf.measures( htf.Measurement("ripple_voltage_mV") .in_range(maximum=50) .with_units(units.MILLIVOLT), ) def phase_output_quality(test): """ATP Section 4.2: Output ripple at full load.""" test.measurements.ripple_voltage_mV = 32.1 @htf.measures( htf.Measurement("leakage_current_uA") .in_range(maximum=500) .with_units(units.MICROAMPERE), ) def phase_safety(test): """ATP Section 4.3: Leakage current per IEC 60950.""" test.measurements.leakage_current_uA = 120.0 ``` ## Step 2: Run the Acceptance Test Connect the test to TofuPilot so every result is logged with the unit's serial number. This creates the traceability record the customer needs. ```python filename="acceptance_test.py" from tofupilot.openhtf import TofuPilot test = htf.Test( phase_power_output, phase_output_quality, phase_safety, ) with TofuPilot(test): test.execute(test_start=lambda: input("Scan unit serial: ")) ``` ## Step 3: Generate Acceptance Records TofuPilot stores every measurement, limit, and pass/fail result per serial number. Open the unit's test history to see: - **Full test record** with all measurements against contractual limits - **Pass/fail status** for each ATP section - **Timestamps** showing exactly when each test ran - **Test reports** exportable for customer review and audits This replaces the manual process of filling out ATP datasheets and scanning signed paper forms. The customer gets the same data in a format that's searchable and auditable. ## Writing a Good ATP A strong acceptance test procedure includes: | Section | Content | |---------|---------| | Scope | What product, what revision, what contract | | Equipment list | Instruments, fixtures, software versions | | Environment | Temperature, humidity, power conditions | | Test steps | Numbered, with exact setup and expected result | | Pass/fail criteria | Numeric limits traceable to contract requirements | | Data recording | What gets recorded, where, and in what format | Keep the ATP stable. Changes require customer approval. Version-control the ATP alongside your test scripts in Git so the test procedure and the code stay in sync. ### What Is Hardware Reliability Testing URL: https://www.tofupilot.com/guides/what-is-hardware-reliability-testing-with-tofupilot Hardware reliability testing predicts product lifespan under stress. Learn what it covers, common methods, and how to track results with TofuPilot. # What Is Hardware Reliability Testing with TofuPilot Hardware reliability testing stresses a product beyond normal conditions to find failure modes before customers do. It predicts how long a product will last and which components fail first. This guide covers the main reliability test methods, how to build reliability test scripts in Python, and how to track results with TofuPilot. ## Why Reliability Testing Matters Production test tells you a unit works right now. Reliability testing tells you it will still work in six months, two years, or ten years. The cost difference between finding a failure mode in the lab versus in the field is typically 10x to 100x. Reliability testing is required or expected in: - Medical devices (FDA 21 CFR Part 820) - Aerospace (DO-160, MIL-STD-810) - Automotive (AEC-Q100/Q200, IATF 16949) - Consumer electronics (IEC 60068, JESD22) ## Common Reliability Test Methods | Method | What It Does | Typical Duration | |--------|-------------|-----------------| | HALT (Highly Accelerated Life Test) | Ramps temperature and vibration until failure | 1-2 weeks | | ALT (Accelerated Life Test) | Runs at elevated stress to predict field life | 2-8 weeks | | Temperature cycling | Alternates hot/cold to stress solder joints | Days to weeks | | Thermal shock | Rapid temperature transitions | Hours to days | | Vibration (random/sine) | Simulates shipping and operational vibration | Hours to days | | Humidity/bias | Powered operation at high humidity | 500-2000 hours | | MTBF verification | Statistical sampling to validate predicted failure rate | Varies | ## HALT vs ALT | Aspect | HALT | ALT | |--------|------|-----| | Goal | Find failure modes | Predict field life | | Stress level | Beyond spec, until failure | Above spec, controlled acceleration | | Sample size | 5-15 units | 20-50 units | | Output | Failure modes and margins | Life prediction with confidence interval | | When | EVT/DVT | DVT/PVT | HALT is qualitative: you're looking for weak spots. ALT is quantitative: you're predicting how long the product lasts. ## Prerequisites - Python 3.10+ - OpenHTF installed (`pip install openhtf`) - TofuPilot Python SDK installed (`pip install tofupilot`) ## Step 1: Define Reliability Check Phases Reliability tests often run a functional check between stress cycles. This detects degradation over time. ```python filename="reliability_check.py" import openhtf as htf from openhtf.util import units @htf.measures( htf.Measurement("output_voltage_V") .in_range( minimum=3.0, maximum=3.6, marginal_minimum=3.1, marginal_maximum=3.5, ) .with_units(units.VOLT), htf.Measurement("current_draw_mA") .in_range(minimum=40, maximum=65) .with_units(units.MILLIAMPERE), ) def phase_functional_check(test): """Run after each stress cycle to detect degradation.""" test.measurements.output_voltage_V = 3.31 test.measurements.current_draw_mA = 49.8 @htf.measures( htf.Measurement("insulation_resistance_MOhm") .in_range(minimum=100) .with_units(units.OHM), ) def phase_insulation_check(test): """Verify insulation resistance hasn't degraded.""" test.measurements.insulation_resistance_MOhm = 450.0 ``` ## Step 2: Log Each Cycle to TofuPilot Run the functional check after every stress cycle and log it as a separate test run. This creates a timeline of measurements per unit that shows degradation trends. ```python filename="reliability_check.py" from tofupilot.openhtf import TofuPilot test = htf.Test( phase_functional_check, phase_insulation_check, ) with TofuPilot(test): test.execute(test_start=lambda: input("Scan unit serial: ")) ``` Run this script after each temperature cycle, vibration session, or time interval. Each execution creates a new test run linked to the same serial number. ## Step 3: Track Degradation in TofuPilot TofuPilot tracks every run per serial number. Open the unit history to see: - **Measurement trends** over time for each unit (voltage drift, current increase) - **Marginal results** flagged before hard failures occur - **Control charts** showing when a parameter starts trending toward a limit - **Failure analysis** identifying which phase fails first across the test lot This data feeds directly into reliability reports. Instead of pulling numbers from chamber logs and multimeter screenshots, the full test history lives in one place. ## When to Use Each Method | Situation | Recommended Method | |-----------|--------------------| | Early prototype, need to find weak spots fast | HALT | | Pre-production, need life prediction for datasheet | ALT | | Qualifying a new supplier or component | Temperature cycling + functional check | | Regulatory submission (medical, aerospace) | Per-standard test (DO-160, IEC 60068) | | Field return investigation | Reproduce failure conditions, run functional checks | Reliability testing is an investment. Start with HALT in EVT to find the obvious failures, then run ALT in DVT to quantify the margins. By PVT, your design should survive the stress levels your customers will see. ### What Is OEE for Test Stations URL: https://www.tofupilot.com/guides/what-is-oee-with-tofupilot Overall equipment effectiveness (OEE) measures manufacturing productivity. Learn how to calculate OEE for test stations and track it with TofuPilot. # What Is OEE with TofuPilot Overall equipment effectiveness (OEE) measures how productively a machine or station is used. It combines three factors: availability, performance, and quality. For test stations, OEE reveals how much of your capacity is actually producing good tested units. This guide covers how to calculate OEE for test stations and how to track its components with TofuPilot. ## The OEE Formula OEE = Availability x Performance x Quality | Factor | What It Measures | Formula | |--------|-----------------|---------| | Availability | How much of planned time the station is running | Run time / Planned production time | | Performance | How fast it runs compared to ideal | (Ideal cycle time x Total units) / Run time | | Quality | How many units pass first time | Good units / Total units | Each factor is a percentage. OEE is their product. ### Example | Factor | Value | Calculation | |--------|-------|------------| | Planned time | 8 hours (480 min) | Shift duration | | Downtime | 45 min (setup + fixture change) | | | Run time | 435 min | 480 - 45 | | Availability | 90.6% | 435 / 480 | | Ideal cycle time | 1 min/unit | | | Total units tested | 400 | | | Performance | 92.0% | (1 x 400) / 435 | | Good units (first pass) | 380 | | | Quality | 95.0% | 380 / 400 | | **OEE** | **79.2%** | 90.6% x 92.0% x 95.0% | World-class OEE is 85%+. Most manufacturing operations run 60-75%. ## The Six Big Losses OEE breaks down into six loss categories. Each maps to one of the three OEE factors: | Loss | OEE Factor | Test Station Example | |------|-----------|---------------------| | Unplanned stops | Availability | Fixture jam, instrument error, PC crash | | Setup and adjustments | Availability | Product changeover, calibration, fixture swap | | Small stops | Performance | Operator absent, DUT loading delay | | Slow cycles | Performance | Instrument settling, retests, slow communication | | Production rejects | Quality | Units that fail test and go to scrap | | Startup rejects | Quality | First units after setup that need retesting | ## Prerequisites - Python 3.10+ - OpenHTF installed (`pip install openhtf`) - TofuPilot Python SDK installed (`pip install tofupilot`) ## Step 1: Capture Test Data Every test run logged to TofuPilot contributes data to the three OEE factors. The test duration feeds performance. The pass/fail result feeds quality. Station uptime feeds availability. ```python filename="production_test.py" import openhtf as htf from openhtf.util import units @htf.measures( htf.Measurement("output_voltage_V") .in_range(minimum=4.9, maximum=5.1) .with_units(units.VOLT), htf.Measurement("current_draw_mA") .in_range(minimum=90, maximum=110) .with_units(units.MILLIAMPERE), ) def phase_functional(test): """Production functional test.""" test.measurements.output_voltage_V = 5.01 test.measurements.current_draw_mA = 99.2 ``` ```python filename="production_test.py" from tofupilot.openhtf import TofuPilot test = htf.Test(phase_functional) with TofuPilot(test): test.execute(test_start=lambda: input("Scan serial: ")) ``` ## Step 2: Track OEE Components in TofuPilot TofuPilot tracks the data you need for each OEE factor: | OEE Factor | TofuPilot Data | |-----------|---------------| | Availability | Station uptime, gaps between test runs | | Performance | Test duration per unit, throughput per hour | | Quality | First pass yield per station | Open the Analytics tab to monitor: - **Station throughput** (units per hour) tracks performance - **First pass yield** tracks quality - **Test gaps** (time between consecutive runs) reveal availability losses ## Improving OEE Focus on the lowest factor first. If availability is 70% but quality is 98%, fixing availability gives the biggest return. | Low Factor | Where to Look | Action | |-----------|--------------|--------| | Availability below 85% | Downtime log, changeover time | Reduce setup time, improve fixture reliability | | Performance below 90% | Phase durations in TofuPilot | Optimize instrument settings, reduce settling time | | Quality below 95% | Failure Pareto in TofuPilot | Fix top failure modes, tighten process controls | Small improvements compound. Improving each factor by 5% (from 85% to 90%) increases OEE from 61.4% to 72.9%. That's 19% more capacity from the same equipment. ### What Is Functional Testing for Hardware URL: https://www.tofupilot.com/guides/what-is-functional-testing-for-hardware-with-tofupilot Functional testing validates that hardware behaves correctly as a complete system. Learn the difference from ICT, how to build functional tests in Python. # What Is Functional Testing for Hardware with TofuPilot Functional testing validates that a hardware product behaves correctly as a complete system. Unlike in-circuit testing (ICT), which checks individual components and solder joints, functional testing treats the board or product as a black box and verifies its inputs and outputs. This guide covers what functional testing involves, how it compares to other test methods, and how to build functional tests in Python with TofuPilot. ## Functional Testing vs Other Methods | Method | What It Tests | How | Finds | |--------|--------------|-----|-------| | ICT (In-Circuit Test) | Individual components | Bed-of-nails fixture, powered | Opens, shorts, wrong values | | Flying probe | Individual components | Moving probes, unpowered/powered | Same as ICT, no fixture needed | | Functional test (FCT) | System behavior | Powered, exercised through interfaces | Integration bugs, firmware issues, performance | | Boundary scan (JTAG) | Digital IC connections | Through JTAG chain | Digital connectivity | ICT tells you the right parts are soldered correctly. Functional testing tells you the product actually works. ## What Functional Tests Cover A functional test exercises the product through its real interfaces: power input, communication ports, sensors, actuators, and user controls. | Category | Example Checks | |----------|---------------| | Power | Startup current, voltage regulation, power sequencing | | Communication | UART, SPI, I2C, CAN, Ethernet respond correctly | | Analog | ADC readings match known inputs within tolerance | | Digital I/O | GPIO states match expected logic levels | | Actuators | Motors, relays, LEDs respond to commands | | Firmware | Version check, self-test pass, boot time | | RF (if applicable) | Transmit power, receive sensitivity, frequency accuracy | ## Prerequisites - Python 3.10+ - OpenHTF installed (`pip install openhtf`) - TofuPilot Python SDK installed (`pip install tofupilot`) ## Step 1: Define Functional Test Phases Each functional check becomes an OpenHTF phase. Group related checks together but keep phases focused enough that a failure points to a specific subsystem. ```python filename="functional_test.py" import openhtf as htf from openhtf.util import units @htf.measures( htf.Measurement("boot_time_ms") .in_range(maximum=2000) .with_units(units.MILLISECOND), htf.Measurement("firmware_version").equals("1.3.0"), ) def phase_power_and_boot(test): """Power up the DUT and verify boot sequence.""" test.measurements.boot_time_ms = 1240 test.measurements.firmware_version = "1.3.0" @htf.measures( htf.Measurement("uart_loopback").equals("PASS"), htf.Measurement("i2c_sensor_id").equals("0x68"), ) def phase_communication(test): """Verify communication interfaces respond correctly.""" test.measurements.uart_loopback = "PASS" test.measurements.i2c_sensor_id = "0x68" @htf.measures( htf.Measurement("adc_channel_0_V") .in_range(minimum=2.45, maximum=2.55) .with_units(units.VOLT), htf.Measurement("adc_channel_1_V") .in_range(minimum=1.60, maximum=1.70) .with_units(units.VOLT), ) def phase_analog_inputs(test): """Apply known voltages and verify ADC readings.""" test.measurements.adc_channel_0_V = 2.50 test.measurements.adc_channel_1_V = 1.65 ``` ## Step 2: Run and Log Results Connect the test to TofuPilot. Every unit's functional test result uploads automatically with full measurement detail. ```python filename="functional_test.py" from tofupilot.openhtf import TofuPilot test = htf.Test( phase_power_and_boot, phase_communication, phase_analog_inputs, ) with TofuPilot(test): test.execute(test_start=lambda: input("Scan serial number: ")) ``` ## Step 3: Track Results in TofuPilot TofuPilot tracks functional test results automatically. Open the Analytics tab to see: - **First pass yield** for each functional test procedure - **Failure Pareto** showing which phases fail most often (power? communication? analog?) - **Measurement distributions** with limit overlays to spot tightening trends - **Station comparison** if you run the same test on multiple stations This data helps you decide where to invest. If 80% of functional test failures come from one phase, that's where to focus your design or process improvement. ## When to Use Functional Testing | Scenario | Use FCT? | |----------|----------| | PCBA with firmware | Yes, after SMT and programming | | Simple passive board | No, ICT or flying probe is sufficient | | Assembled product with sensors/actuators | Yes, as EOL test | | Safety-critical product | Yes, with documented test procedure | | High-volume, low-complexity | Maybe, depends on field failure cost | Functional testing adds cycle time, but it catches the integration failures that ICT cannot. For products with firmware, communication interfaces, or analog circuits, it's the test stage that matters most. ### Deploy Test Scripts to Production URL: https://www.tofupilot.com/guides/how-to-deploy-python-test-scripts-to-production-stations Learn how to package, distribute, and deploy OpenHTF test scripts to production stations using virtual environments, PyInstaller, and version tracking. Deploying test scripts to production stations requires a repeatable process: pin dependencies, bundle the executable, configure each station's environment, and track versions in TofuPilot. This guide covers the full workflow from a clean virtual environment to a versioned deployment across multiple stations. ## Prerequisites - Python 3.9 or later installed on all target stations - TofuPilot account with at least one station configured - `tofupilot` and `openhtf` packages available on PyPI ## Step 1: Set Up a Virtual Environment Isolate your test script dependencies from the system Python to avoid version conflicts across deployments. ```bash filename="setup_env.sh" #!/bin/bash python3 -m venv .venv source .venv/bin/activate pip install --upgrade pip ``` On Windows stations: ```bat filename="setup_env.bat" @echo off python -m venv .venv .venv\Scripts\activate pip install --upgrade pip ``` ## Step 2: Pin Dependencies Pin all dependency versions to ensure every station runs the same code. Unpinned dependencies are the most common source of "works on my machine" failures in production. ```text filename="requirements.txt" openhtf==2.1.0 tofupilot==1.5.0 pyserial==3.5 numpy==1.26.4 pyinstaller==6.10.0 ``` Install and verify: ```bash filename="install_deps.sh" #!/bin/bash pip install -r requirements.txt pip freeze > requirements.lock.txt # capture transitive deps for audit ``` Use `requirements.lock.txt` for auditing; use `requirements.txt` for installs. Never pin transitive dependencies manually. ### Using pyproject.toml (Optional) If you manage your test scripts as a package, use `pyproject.toml` instead: ```toml filename="pyproject.toml" [project] name = "station-tests" version = "1.4.2" requires-python = ">=3.9" dependencies = [ "openhtf==2.1.0", "tofupilot==1.5.0", "pyserial==3.5", "numpy==1.26.4", ] [build-system] requires = ["setuptools>=68"] build-backend = "setuptools.backends.legacy:build" ``` The `version` field in `pyproject.toml` becomes your deployment version. Increment it before each release. ## Step 3: Write the Test Script Structure your test with plug injection using the `@htf.plug` decorator. Don't use type hints for plug injection. ```python filename="test_main.py" import openhtf as htf from tofupilot.openhtf import TofuPilot from plugs.serial_plug import SerialPlug from plugs.power_plug import PowerPlug @htf.measures( htf.Measurement("boot_time_ms") .in_range(maximum=5000) .doc("Time from power-on to READY response"), ) @htf.plug(serial=SerialPlug) @htf.plug(power=PowerPlug) def phase_power_on(test, serial, power): power.enable() response = serial.read_until("READY", timeout_s=5) test.measurements.boot_time_ms = response.elapsed_ms @htf.measures( htf.Measurement("voltage_mv") .in_range(minimum=4900, maximum=5100) ) @htf.plug(power=PowerPlug) def phase_voltage_check(test, power): voltage = power.measure_voltage() test.measurements.voltage_mv = voltage def main(): test = htf.Test( phase_power_on, phase_voltage_check, test_name="PCB Functional Test", ) with TofuPilot(test): test.execute(test_start=lambda: input("Enter serial number: ").strip()) if __name__ == "__main__": main() ``` ## Step 4: Bundle with PyInstaller PyInstaller packages the script and all dependencies into a single executable. This eliminates Python version mismatches on stations and simplifies deployment to one file copy. PyInstaller can't detect OpenHTF's dynamic imports automatically, so you need explicit hook configuration. The spec file below handles the known hidden imports. ```python filename="station_tests.spec" # -*- mode: python ; coding: utf-8 -*- import sys from PyInstaller.utils.hooks import collect_data_files, collect_submodules block_cipher = None # OpenHTF uses dynamic imports that PyInstaller can't detect automatically openhtf_datas = collect_data_files("openhtf") openhtf_hiddenimports = collect_submodules("openhtf") a = Analysis( ["test_main.py"], pathex=["."], binaries=[], datas=openhtf_datas + [ ("plugs/", "plugs/"), ("config/", "config/"), ("VERSION", "."), ], hiddenimports=openhtf_hiddenimports + [ "openhtf.output.callbacks", "openhtf.output.callbacks.json_factory", "openhtf.util.logs", "tofupilot", "tofupilot.openhtf", ], hookspath=[], hooksconfig={}, runtime_hooks=[], excludes=[], win_no_prefer_redirects=False, win_private_assemblies=False, cipher=block_cipher, noarchive=False, ) pyz = PYZ(a.pure, a.zipped_data, cipher=block_cipher) exe = EXE( pyz, a.scripts, a.binaries, a.zipfiles, a.datas, [], name="station_tests", debug=False, bootloader_ignore_signals=False, strip=False, upx=False, upx_exclude=[], runtime_tmpdir=None, console=True, disable_windowed_traceback=False, target_arch=None, codesign_identity=None, entitlements_file=None, ) ``` Build the executable: ```bash filename="build.sh" #!/bin/bash set -e VERSION=$(python -c "import tomllib; print(tomllib.load(open('pyproject.toml','rb'))['project']['version'])") echo "$VERSION" > VERSION echo "Building version $VERSION" pyinstaller station_tests.spec --clean --noconfirm mv dist/station_tests "dist/station_tests_v${VERSION}" rm VERSION echo "Built: dist/station_tests_v${VERSION}" ``` ### PyInstaller Troubleshooting | Symptom | Cause | Fix | |---|---|---| | `ModuleNotFoundError: openhtf.util.logs` | Missing hidden import | Add to `hiddenimports` in spec | | Blank frontend at `localhost:4444` | openhtf web assets not bundled | Check `collect_data_files("openhtf")` in spec | | `OSError: [Errno 2]` on config file | Data file not included | Add config dir to `datas` in spec | | Crash on second run | Leftover `_MEIPASS` temp dir | Set `runtime_tmpdir` to a fixed path | ## Step 5: Configure Station Environment Each station needs its own identity and credentials. Never hardcode station IDs or API keys in the script. ### Environment Variables (Recommended) ```bash filename="station_env.sh" #!/bin/bash # Source this file on each station before running tests # Place in /etc/profile.d/tofupilot.sh for persistent configuration export TOFUPILOT_API_KEY="tp_live_xxxxxxxxxxxxxxxxxxxx" export TOFUPILOT_STATION_ID="station-floor-1-cell-3" export STATION_SERIAL_PORT="/dev/ttyUSB0" export STATION_BAUD_RATE="115200" ``` ### Config File Fallback For stations where environment variables are impractical, use a local config file: ```json filename="/etc/tofupilot/station.json" { "api_key": "tp_live_xxxxxxxxxxxxxxxxxxxx", "station_id": "station-floor-1-cell-3", "serial_port": "/dev/ttyUSB0", "baud_rate": 115200 } ``` Load it in the script: ```python filename="config/loader.py" import json import os from pathlib import Path def load_station_config() -> dict: if os.environ.get("TOFUPILOT_API_KEY"): return { "api_key": os.environ["TOFUPILOT_API_KEY"], "station_id": os.environ.get("TOFUPILOT_STATION_ID", "unknown"), "serial_port": os.environ.get("STATION_SERIAL_PORT", "/dev/ttyUSB0"), "baud_rate": int(os.environ.get("STATION_BAUD_RATE", "115200")), } config_path = Path("/etc/tofupilot/station.json") if not config_path.exists(): raise FileNotFoundError( f"Station config not found at {config_path}. " "Set TOFUPILOT_API_KEY environment variable or create the config file." ) with config_path.open() as f: return json.load(f) ``` ## Step 6: Deploy to Multiple Stations Use a deploy script to push the executable and config in one step. This example targets Linux stations over SSH. ```bash filename="deploy.sh" #!/bin/bash set -e VERSION=$(python -c "import tomllib; print(tomllib.load(open('pyproject.toml','rb'))['project']['version'])") EXECUTABLE="dist/station_tests_v${VERSION}" DEPLOY_PATH="/opt/tofupilot/station_tests" STATIONS=( "operator@station-01.local" "operator@station-02.local" "operator@station-03.local" ) for STATION in "${STATIONS[@]}"; do echo "Deploying v${VERSION} to ${STATION}..." ssh "$STATION" "[ -f ${DEPLOY_PATH} ] && cp ${DEPLOY_PATH} ${DEPLOY_PATH}.bak || true" scp "$EXECUTABLE" "${STATION}:${DEPLOY_PATH}" ssh "$STATION" "chmod +x ${DEPLOY_PATH}" ssh "$STATION" "${DEPLOY_PATH} --version 2>&1 | head -1" echo " Done: ${STATION}" done echo "Deployed v${VERSION} to ${#STATIONS[@]} stations." ``` ## Step 7: Track Versions in TofuPilot TofuPilot records the software version on every test run. Set it explicitly so you can filter runs by deployment version and correlate yield changes with code releases. ```python filename="test_main.py" import importlib.metadata import openhtf as htf from tofupilot.openhtf import TofuPilot def get_version() -> str: try: return importlib.metadata.version("station-tests") except importlib.metadata.PackageNotFoundError: import os, sys base = getattr(sys, "_MEIPASS", os.path.dirname(__file__)) version_file = os.path.join(base, "VERSION") with open(version_file) as f: return f.read().strip() def main(): test = htf.Test( phase_power_on, phase_voltage_check, test_name="PCB Functional Test", ) with TofuPilot(test, software_version=get_version()): test.execute(test_start=lambda: input("Enter serial number: ").strip()) ``` ### Deployment Comparison | Method | Setup | Station Python required | Rollback | |---|---|---|---| | Virtual environment | Low | Yes | Replace venv | | PyInstaller single executable | Medium | No | Replace file | | Docker container | High | No | `docker pull` previous tag | | Network share (live) | Low | Yes | Revert file on share | PyInstaller is the right choice when stations have locked-down OS images or inconsistent Python versions. Virtual environments work well when you control the station OS and want faster iteration. ### How to Configure Test Environments URL: https://www.tofupilot.com/guides/how-to-configure-test-environments-for-dev-qa-and-production Learn how to manage separate test environments with environment variables, config files, and TofuPilot workspace separation. Your test code shouldn't change between environments. The same script that runs on your dev bench needs to run in QA validation and on the production floor. What changes is the configuration: instrument addresses, measurement limits, TofuPilot workspace, logging levels. This guide shows three approaches to environment-specific config and how to point each environment at its own TofuPilot workspace. ## The Problem A single test script typically needs different values for: - **Instrument addresses.** Your dev bench DMM is at `TCPIP::192.168.1.10` but production uses `GPIB0::22`. - **Measurement limits.** Dev uses wider tolerances for debugging. Production uses datasheet limits. - **TofuPilot workspace.** Dev runs shouldn't pollute production data. - **Logging verbosity.** Debug logging in dev, errors only in production. - **Fixture IDs.** Each station has its own fixture with unique calibration data. Hardcoding any of these means editing code for every deployment. That's how bugs reach the production floor. ## Approach 1: Environment Variables with python-dotenv The simplest approach. Create a `.env` file per environment and load it at startup. ```python filename=".env.dev" TOFUPILOT_API_KEY=tp_dev_abc123 DMM_ADDRESS=TCPIP::192.168.1.10::INSTR PSU_ADDRESS=TCPIP::192.168.1.11::INSTR VOLTAGE_MIN=3.0 VOLTAGE_MAX=3.6 LOG_LEVEL=DEBUG STATION_ID=DEV-BENCH-01 ``` ```python filename=".env.production" TOFUPILOT_API_KEY=tp_prod_xyz789 DMM_ADDRESS=GPIB0::22::INSTR PSU_ADDRESS=GPIB0::5::INSTR VOLTAGE_MIN=3.2 VOLTAGE_MAX=3.4 LOG_LEVEL=ERROR STATION_ID=PROD-LINE1-STN04 ``` Load the right file based on an environment flag: ```python filename="config.py" import os from dotenv import load_dotenv # Set TEST_ENV=dev|qa|production before running env = os.getenv("TEST_ENV", "dev") load_dotenv(f".env.{env}") class Config: TOFUPILOT_API_KEY = os.getenv("TOFUPILOT_API_KEY") DMM_ADDRESS = os.getenv("DMM_ADDRESS") PSU_ADDRESS = os.getenv("PSU_ADDRESS") VOLTAGE_MIN = float(os.getenv("VOLTAGE_MIN", "3.2")) VOLTAGE_MAX = float(os.getenv("VOLTAGE_MAX", "3.4")) LOG_LEVEL = os.getenv("LOG_LEVEL", "INFO") STATION_ID = os.getenv("STATION_ID", "UNKNOWN") ``` Use it in your test: ```python filename="test_power.py" import openhtf as htf from tofupilot.openhtf import TofuPilot from config import Config @htf.measures( htf.Measurement("voltage_3v3").in_range(Config.VOLTAGE_MIN, Config.VOLTAGE_MAX), ) def test_power_rail(test): # Use Config.DMM_ADDRESS to connect to the right instrument test.measurements.voltage_3v3 = 3.29 def main(): test = htf.Test(test_power_rail) with TofuPilot(test): test.execute(test_start=lambda: "DUT-001") if __name__ == "__main__": main() ``` Run with: `TEST_ENV=production python test_power.py` ## Approach 2: YAML Config Files For more complex configurations with nested structures, YAML is cleaner than flat env vars. ```python filename="config/production.yaml" instruments: dmm: address: "GPIB0::22::INSTR" timeout_ms: 5000 psu: address: "GPIB0::5::INSTR" timeout_ms: 3000 limits: voltage_3v3: min: 3.2 max: 3.4 current_idle: max: 0.050 station: id: "PROD-LINE1-STN04" log_level: "ERROR" ``` ```python filename="config/dev.yaml" instruments: dmm: address: "TCPIP::192.168.1.10::INSTR" timeout_ms: 10000 psu: address: "TCPIP::192.168.1.11::INSTR" timeout_ms: 10000 limits: voltage_3v3: min: 3.0 max: 3.6 current_idle: max: 0.100 station: id: "DEV-BENCH-01" log_level: "DEBUG" ``` Load with a simple helper: ```python filename="config_loader.py" import os import yaml from pathlib import Path def load_config(env: str = None) -> dict: """Load YAML config for the given environment.""" env = env or os.getenv("TEST_ENV", "dev") config_path = Path(__file__).parent / "config" / f"{env}.yaml" if not config_path.exists(): raise FileNotFoundError(f"No config file for environment: {env}") with open(config_path) as f: return yaml.safe_load(f) ``` ```python filename="test_with_yaml.py" import openhtf as htf from tofupilot.openhtf import TofuPilot from config_loader import load_config cfg = load_config() limits = cfg["limits"]["voltage_3v3"] @htf.measures( htf.Measurement("voltage_3v3").in_range(limits["min"], limits["max"]), ) def test_power_rail(test): test.measurements.voltage_3v3 = 3.29 def main(): test = htf.Test(test_power_rail) with TofuPilot(test): test.execute(test_start=lambda: "DUT-001") if __name__ == "__main__": main() ``` ## Point to Different TofuPilot Workspaces Each environment should write to its own TofuPilot workspace. This keeps dev noise out of production data. Set the `TOFUPILOT_API_KEY` environment variable per environment. Each API key is scoped to a workspace. No code changes needed. ```python filename="run_test.sh" # Dev export TOFUPILOT_API_KEY="tp_dev_abc123" python test_power.py # QA export TOFUPILOT_API_KEY="tp_qa_def456" python test_power.py # Production export TOFUPILOT_API_KEY="tp_prod_xyz789" python test_power.py ``` The TofuPilot client reads `TOFUPILOT_API_KEY` from the environment automatically. You don't need to pass it in code. ## Comparison: Env Vars vs Config Files vs CLI Args | Approach | Best for | Pros | Cons | |----------|----------|------|------| | Environment variables | Simple configs, CI/CD, secrets | No files to deploy, works with Docker/systemd, secret-friendly | Flat structure only, no nesting | | YAML config files | Complex configs, nested structures | Readable, supports comments, version-controllable | Need to deploy files, secrets in plaintext | | CLI arguments | One-off overrides, debugging | No files needed, easy to test | Verbose for many params, not persistent | | Combined (env + YAML) | Production deployments | Secrets in env, structure in YAML | Two systems to maintain | The pragmatic approach: use YAML for instrument addresses and limits (things that vary by station), and environment variables for secrets (API keys, credentials). CLI args for quick overrides during debugging. ### Track PCBA Results Across Lines URL: https://www.tofupilot.com/guides/how-to-track-pcba-test-results-across-production-lines-with-tofupilot Learn how to tag OpenHTF test runs with station, line, factory, and shift metadata, then compare first pass yield across production lines using TofuPilot. TofuPilot associates every test run with a named station. By encoding line, factory, and shift into the station name and run metadata, you get a single query surface that spans every production line and factory without custom tooling. ## Why Multi-Line Traceability Matters When a PCBA defect appears in the field, you need to answer three questions fast: - Which production line built the affected units? - Which shift was running at the time? - Is the failure isolated to one station or systemic across a line? Without structured metadata on each run, answering these questions means cross-referencing operator logs, shift schedules, and test CSV exports by hand. TofuPilot solves this by making station identity and run metadata first-class fields on every test record. | Without structured metadata | With TofuPilot station and run metadata | |---|---| | Manual log cross-referencing | Single dashboard filter | | Shift data in spreadsheets | Encoded in run at test time | | Per-line export files | Unified API query | | Yield comparison in Excel | Built-in per-station yield chart | ## Prerequisites - Python 3.9+ - OpenHTF installed (`pip install openhtf`) - TofuPilot client installed (`pip install tofupilot`) - A TofuPilot account with at least one procedure created ## Setting Up Station Identity in TofuPilot Each physical test station maps to a named station in TofuPilot. The naming convention carries all the traceability context you need. ### Station Naming Convention Use a structured name that encodes factory, line, and station number: ``` {FACTORY}-{LINE}-FCT{STATION_NUMBER} ``` Examples: | Station name | Factory | Line | Station | |---|---|---|---| | `SZX-L1-FCT01` | Shenzhen | Line 1 | FCT station 1 | | `SZX-L2-FCT01` | Shenzhen | Line 2 | FCT station 1 | | `TXL-L1-FCT01` | Toulouse | Line 1 | FCT station 1 | | `TXL-L1-FCT02` | Toulouse | Line 1 | FCT station 2 | ### Environment-Based Station Configuration Store the station identity in environment variables on each machine, not in the test script: ```bash filename="/etc/environment" TOFUPILOT_API_KEY=tp_station_xxxxxxxxxxxxx STATION_ID=SZX-L1-FCT01 FACTORY=SZX LINE=L1 ``` Load them in your test script at runtime: ```python filename="config.py" import os STATION_ID = os.environ["STATION_ID"] FACTORY = os.environ["FACTORY"] LINE = os.environ["LINE"] ``` This approach means the same test script binary deploys to every station. Only the environment differs. ## Tagging Runs with Line, Factory, and Shift Metadata ### Determining the Current Shift ```python filename="shift.py" from datetime import datetime def get_current_shift() -> str: """Return shift label based on local wall clock.""" hour = datetime.now().hour if 6 <= hour < 14: return "morning" elif 14 <= hour < 22: return "afternoon" else: return "night" ``` ### Full OpenHTF Test with Station Metadata This example tests a PCBA power supply board. The station identity, line, factory, and shift are injected at test setup time and attached to every run. ```python filename="pcba_fct_test.py" import os import openhtf as htf from openhtf.util import units from tofupilot.openhtf import TofuPilot import config from shift import get_current_shift class PowerSupplyPlug(htf.plugs.BasePlug): """Controls the bench PSU over USB-serial.""" def setUp(self): import serial self._port = serial.Serial("/dev/ttyUSB0", 9600, timeout=1) def set_voltage(self, volts: float, channel: int = 1): self._port.write(f":APPL CH{channel},{volts},1.0\n".encode()) def measure_voltage(self, channel: int = 1) -> float: self._port.write(f":MEAS:VOLT? CH{channel}\n".encode()) return float(self._port.readline().strip()) def tearDown(self): self._port.close() @htf.plug(psu=PowerSupplyPlug) @htf.measures( htf.Measurement("rail_3v3") .in_range(minimum=3.235, maximum=3.365) .with_units(units.VOLT) .doc("3.3 V rail under 100 mA load"), htf.Measurement("rail_5v0") .in_range(minimum=4.900, maximum=5.100) .with_units(units.VOLT) .doc("5.0 V rail under 200 mA load"), ) def test_power_rails(test, psu): psu.set_voltage(3.3, channel=1) test.measurements.rail_3v3 = psu.measure_voltage(channel=1) psu.set_voltage(5.0, channel=2) test.measurements.rail_5v0 = psu.measure_voltage(channel=2) @htf.plug(psu=PowerSupplyPlug) @htf.measures( htf.Measurement("idle_current") .in_range(minimum=0, maximum=0.350) .with_units(units.AMPERE) .doc("Total board current draw at idle"), ) def test_idle_current(test, psu): psu.set_voltage(5.0, channel=2) current_a = float(psu._port.readline().strip()) test.measurements.idle_current = current_a def main(): serial_number = input("Scan DUT serial number: ").strip() test = htf.Test( test_power_rails, test_idle_current, procedure_id="PCBA-FCT-001", ) with TofuPilot(test): test.execute(test_start=lambda: serial_number) if __name__ == "__main__": main() ``` Every run in TofuPilot carries the station identity as queryable metadata alongside the standard pass/fail result and measurements. ## Comparing Results Across Lines in TofuPilot With runs uploading from all lines, TofuPilot's filtering and analytics give you direct comparison: - **FPY by station** shows yield for each station grouped by line prefix. If Line B consistently trails Line A, the problem is systemic to that line. - **Measurement histograms** reveal whether one line's values are shifted or have wider spread. A shifted mean suggests calibration offset. - **Failure Pareto by station** shows which specific tests fail more often on each station. If one station accounts for most failures, start investigating that fixture. - **Trend charts** show whether yield gaps are constant, growing, or appeared suddenly after a change. ## Investigating Yield Differences When you find a yield gap between lines, narrow down the cause systematically: 1. **Check measurement distributions.** If one line's values are offset, it's likely calibration or equipment. If they're wider, it's process variation. 2. **Check by time of day.** Yield drops on night shifts point to operator training or environmental changes. 3. **Check individual stations.** Sometimes the "line" problem is actually one bad station dragging down the average. 4. **Check by component lot.** If you track lot numbers as metadata, filter by lot to see if specific batches drive the difference. A station with FPY below 95% while neighboring stations are at 97-98% typically indicates a fixture contact issue, cable degradation, or calibration drift. ## Deployment Checklist | Step | Action | |---|---| | Station naming | Follow `{FACTORY}-{LINE}-FCT{N}` convention | | API keys | One key per station, stored in environment | | Procedure ID | Same ID across all lines for unified yield view | | Dashboard | Verify station names appear on first run before full rollout | ### Scale from 1 to 100 Test Stations URL: https://www.tofupilot.com/guides/how-to-scale-from-1-to-100-test-stations-with-tofupilot Architecture guide for scaling test infrastructure from 1 to 100 stations, covering naming, script distribution, config management, networking, and monitoring. Run one station or a hundred: TofuPilot handles both, but the architecture decisions you make at 10 stations determine whether you succeed at 100. This guide walks through the changes needed at each scale threshold. ## Scaling Challenges by Stage | Stations | What breaks | Root cause | |----------|-------------|------------| | 1 | Nothing | You're fine | | 5-10 | Config drift | Manual setup per station | | 10-25 | Script version mismatch | No deployment pipeline | | 25-50 | Debugging blindness | No centralized logs | | 50-100 | Network saturation | All stations hit cloud simultaneously | | 100+ | Identity collision | Non-unique station names | ## Architecture at Each Scale ### 1 Station A single machine running your test script. No special infrastructure needed. ``` [DUT] -> [Test Station] -> [TofuPilot Cloud] ``` ### 10 Stations At 10 stations, the bottleneck is configuration. You need a shared config source, consistent naming, and a way to push script updates. ``` [10 DUTs] -> [10 Test Stations] -> [TofuPilot Cloud] | [Git repo for scripts] ``` ### 100 Stations At 100, you need a deployment pipeline, centralized monitoring, and staggered upload scheduling. ``` [100 DUTs] -> [100 Test Stations] -> [TofuPilot Cloud] | | [CI/CD pipeline] [Health dashboard] [Config server] ``` ## Station Naming and Registration TofuPilot identifies stations by name. A collision corrupts per-station analytics. ``` {site}-{line}-{station_number} # Examples: taipei-line1-001 munich-eol-001 ``` Set the station name as an environment variable: ```bash filename="/etc/environment" TOFUPILOT_STATION_ID=taipei-line1-001 TOFUPILOT_API_KEY=tp_live_xxxxxxxxxxxx ``` ## Test Script Distribution ### Option 1: Git Pull (1-20 stations) ```bash filename="/etc/cron.d/update-test-script" 0 6 * * * testuser cd /opt/testscripts && git pull origin main ``` Pros: Simple. Cons: Requires network access to Git host. Fails silently. ### Option 2: PyInstaller Binary (20-50 stations) ```bash filename="build_and_deploy.sh" #!/bin/bash pyinstaller --onefile test_main.py --name test_runner for station in taipei-line1-{001..020}; do rsync -az dist/test_runner "${station}:/opt/testscripts/test_runner_new" ssh "${station}" "mv /opt/testscripts/test_runner_new /opt/testscripts/test_runner" done ``` Pros: No Python on stations. Cons: Slower iteration, larger binary. ### Option 3: Docker (50-100 stations) ```dockerfile filename="Dockerfile" FROM python:3.11-slim WORKDIR /app COPY requirements.txt . RUN pip install --no-cache-dir -r requirements.txt COPY test_main.py . CMD ["python", "test_main.py"] ``` ```yaml filename="docker-compose.yml" version: "3.9" services: test_runner: image: your-registry/test-runner:latest restart: unless-stopped environment: - TOFUPILOT_STATION_ID - TOFUPILOT_API_KEY devices: - /dev/ttyUSB0:/dev/ttyUSB0 ``` Update all stations: ```bash filename="deploy.sh" #!/bin/bash parallel-ssh -h stations.txt "docker compose pull && docker compose up -d" ``` ### Distribution Method Comparison | Method | Stations | Python on station | Update speed | Rollback | |--------|----------|-------------------|-------------|----------| | Git pull | 1-20 | Yes | Fast | `git revert` | | PyInstaller | 20-50 | No | Medium | Replace binary | | Docker | 50-100 | No | Fast | Previous image tag | ## Centralized Configuration Never hardcode values that vary per station or product. | Config tier | Scope | Example | |-------------|-------|---------| | Environment variable | Per station | `TOFUPILOT_STATION_ID`, `TOFUPILOT_API_KEY` | | Config file | Per product | Voltage limits, timing thresholds | | Test script | Per test phase | OpenHTF phase logic | ```yaml filename="config/product_v2.yaml" voltage_rail_3v3: min: 3.2 max: 3.4 voltage_rail_5v: min: 4.8 max: 5.2 boot_time_ms: min: 0 max: 3000 ``` ```python filename="test_main.py" import yaml import openhtf as htf from openhtf.util import units from tofupilot.openhtf import TofuPilot with open("config/product_v2.yaml") as f: cfg = yaml.safe_load(f) @htf.measures( htf.Measurement("voltage_3v3").in_range( cfg["voltage_rail_3v3"]["min"], cfg["voltage_rail_3v3"]["max"] ).with_units(units.VOLT) ) def phase_power_check(test): voltage = read_voltage_rail("3v3") test.measurements.voltage_3v3 = voltage ``` ## Network Topology Stations communicate only outbound to TofuPilot. No inbound ports required. | Destination | Port | Purpose | |-------------|------|---------| | `tofupilot.app` | 443 | Test result upload | | Your Git host | 443 | Script updates | | Your container registry | 443 | Image pulls | At 100 stations, stagger uploads to avoid thundering herd: ```python filename="upload_utils.py" import time import random def upload_with_jitter(upload_fn, max_jitter_seconds=10): time.sleep(random.uniform(0, max_jitter_seconds)) upload_fn() ``` ## Monitoring Station Health TofuPilot tracks per-station yield, throughput, and test duration automatically. Open the Analytics tab filtered by station to spot degradation. ### Station Health Checklist | Check | Interval | Action on failure | |-------|----------|-------------------| | Heartbeat to TofuPilot | 1 min | Alert on-call, check network | | Test script version | On start | Auto-update via pipeline | | Disk space | 1 hour | Archive old logs, alert if under 5 GB | | USB fixture connectivity | Before each run | Fail test with clear error | ## Station Bootstrap Script ```bash filename="bootstrap.sh" #!/bin/bash set -e STATION_ID=$1 API_KEY=$2 if [ -z "$STATION_ID" ] || [ -z "$API_KEY" ]; then echo "Usage: bootstrap.sh " exit 1 fi echo "TOFUPILOT_STATION_ID=${STATION_ID}" >> /etc/environment echo "TOFUPILOT_API_KEY=${API_KEY}" >> /etc/environment curl -fsSL https://get.docker.com | sh mkdir -p /opt/testrunner cat > /opt/testrunner/docker-compose.yml << EOF version: "3.9" services: test_runner: image: your-registry/test-runner:latest restart: unless-stopped env_file: /etc/environment EOF docker compose -f /opt/testrunner/docker-compose.yml up -d echo "Station ${STATION_ID} is running." ``` ```bash filename="terminal" sudo bash bootstrap.sh taipei-line1-042 tp_live_xxxxxxxxxxxx ``` ### How to Optimize Manufacturing Test Stations URL: https://www.tofupilot.com/guides/how-to-optimize-test-stations-with-tofupilot Test station optimization reduces cycle time and increases throughput. Learn how to identify bottlenecks and improve station performance with TofuPilot. # How to Optimize Test Stations with TofuPilot Test station cycle time directly limits production throughput. A 60-second test on two stations means 120 units per hour. Cut that to 45 seconds and you're at 160 without adding hardware. This guide covers how to find bottlenecks, reduce cycle time, and track station performance with TofuPilot. ## Where Time Goes A typical test station cycle breaks down into: | Phase | Typical Share | Example | |-------|--------------|---------| | Fixture load/unload | 20-30% | Operator places DUT, closes clamp | | Instrument settling | 10-20% | DMM auto-range, power supply ramp | | Measurement acquisition | 30-40% | Read voltage, current, frequency | | Data logging and reporting | 5-10% | Write results to database | | Idle/wait | 5-15% | Operator between units | The biggest gains usually come from instrument settling and measurement acquisition, not from faster code. ## Step 1: Measure Your Baseline You can't optimize what you don't measure. OpenHTF records the duration of every phase automatically. TofuPilot displays these timings in the test step performance view. ```python filename="optimized_test.py" import openhtf as htf from openhtf.util import units @htf.measures( htf.Measurement("supply_voltage_V") .in_range(minimum=4.9, maximum=5.1) .with_units(units.VOLT), ) def phase_power_check(test): """Measure supply voltage after power-up.""" test.measurements.supply_voltage_V = 5.01 @htf.measures( htf.Measurement("signal_amplitude_V") .in_range(minimum=1.8, maximum=2.2) .with_units(units.VOLT), ) def phase_signal_check(test): """Measure signal output amplitude.""" test.measurements.signal_amplitude_V = 2.01 @htf.measures( htf.Measurement("self_test_result").equals("PASS"), ) def phase_firmware_self_test(test): """Run firmware self-test and check result.""" test.measurements.self_test_result = "PASS" ``` Run this on 50 units and review the phase durations in TofuPilot. The longest phase is your bottleneck. ## Step 2: Reduce Instrument Settling Time Instrument settling is often the hidden time sink. Auto-ranging, input impedance switching, and filter settling add hundreds of milliseconds per measurement. | Optimization | Time Saved | How | |-------------|-----------|-----| | Fixed range instead of auto-range | 100-500ms per reading | Set range in SCPI before measuring | | Reduce integration time (NPLC) | 50-200ms per reading | Lower NPLC from 10 to 1 (trades accuracy) | | Minimize channel switching | 50-100ms per switch | Group measurements by channel | | Parallel instrument commands | Varies | Send setup commands while another instrument reads | Only reduce accuracy where the measurement margin allows it. If your limit is 4.9V to 5.1V and units read 5.0V with 1mV spread, you have room to reduce NPLC. ## Step 3: Optimize Test Sequence Phase order matters. Put the fastest-failing tests first to reduce average cycle time on bad units. | Strategy | Effect | |----------|--------| | Fail-fast ordering | Tests that catch 80% of defects run first | | Power-up checks before functional | Catch dead boards before slow tests | | Skip downstream phases on failure | Don't test analog if power-up failed | ```python filename="optimized_test.py" from openhtf import PhaseResult @htf.measures( htf.Measurement("power_good").equals("PASS"), ) def phase_power_good(test): """Quick power check. If this fails, skip everything else.""" result = "PASS" test.measurements.power_good = result if result != "PASS": return PhaseResult.STOP ``` ## Step 4: Track Station Performance Connect the test to TofuPilot and monitor station-level metrics over time. ```python filename="optimized_test.py" from tofupilot.openhtf import TofuPilot test = htf.Test( phase_power_good, phase_power_check, phase_signal_check, phase_firmware_self_test, ) with TofuPilot(test): test.execute(test_start=lambda: input("Scan serial: ")) ``` TofuPilot tracks station performance automatically. Open the Analytics tab to see: - **Throughput per station** (units per hour) - **Phase duration breakdown** showing where time is spent - **Yield per station** to catch fixture or equipment issues - **Station comparison** to identify underperforming stations ## Optimization Checklist | Check | Action | |-------|--------| | Longest phase identified | Review in TofuPilot step performance | | Auto-range disabled where possible | Set fixed range via SCPI | | NPLC reduced where margin allows | Check measurement distributions | | Fail-fast order implemented | Move high-failure phases earlier | | Phase skip on critical failure | Return PhaseResult.STOP | | Fixture load time measured | Time the operator, not just the code | Station optimization is iterative. Make one change, measure the effect in TofuPilot, then move to the next bottleneck. Small gains compound: five 10% improvements give you a 40% faster station. ### Design for Testability: PCB Best Practices URL: https://www.tofupilot.com/guides/design-for-testability-pcb-best-practices-with-tofupilot PCB design for testability (DFT) guidelines covering test point placement, JTAG boundary scan, programming headers, and how DFT enables automated testing. Good DFT decisions made during PCB layout directly reduce test time, fixture cost, and escapes at ICT, functional test, and in-system programming. A board that's hard to probe is hard to test automatically. This guide covers the mechanical and electrical decisions that make automated testing with OpenHTF and TofuPilot practical. ## What DFT Means for PCBAs Design for Testability (DFT) is the set of layout and schematic decisions that make a PCBA observable and controllable during manufacturing test. | Property | Definition | Test impact | |----------|-----------|-------------| | Observability | Can you measure the signal? | Missing test points force blind assumptions | | Controllability | Can you force the circuit into a known state? | Missing isolation makes component-level testing impossible | | Accessibility | Can the probe physically reach the pad? | Covered vias and undersized pads break fixturing | ## Test Point Placement Guidelines Test points are the interface between your board and the test fixture. Every net you want to probe at ICT or functional test needs one. ### Physical Requirements | Parameter | Minimum | Preferred | |-----------|---------|-----------| | Test point diameter | 0.9 mm (35 mil) | 1.27 mm (50 mil) | | Pad-to-pad spacing | 1.27 mm center-to-center | 2.54 mm (100 mil grid) | | Clearance from components | 0.5 mm | 1.0 mm | | Board edge clearance | 3.0 mm | 5.0 mm | | Max probe travel (spring pin) | 3.0 mm | 1.5 mm nominal | Place test points on a 100 mil grid where possible. Bed-of-nails fixtures are built on that grid. ### Which Nets Need Test Points Cover these at minimum: - Power rails (VCC, VBAT, VIO, all regulator outputs) - Ground (one per power domain) - Digital I/O on microcontrollers and FPGAs - Analog signal paths (input and output) - Crystal oscillator pins - Reset lines - Communication buses (UART TX/RX, SPI CS/CLK/MOSI/MISO, I2C SDA/SCL) - Programming and debug interfaces (SWDIO, SWDCLK, TDI, TDO, TMS, TCK) ### Placement Rules Put test points on the primary side (component side) when possible. Double-sided fixtures are expensive and slow cycle time. If a net only routes on the back side, add a via to the top with a test point. Do not cover test point vias with soldermask. Soldermask-covered vias are common for density but kill ICT access. ## Common DFT Mistakes and Test Cost Impact | Mistake | How it manifests | Cost impact | |---------|-----------------|-------------| | Test points on bottom side only | Requires double-sided fixture | 2x fixture cost, slower cycle time | | Test points under connectors or heatsinks | Unreachable by spring pin | Net is untestable without rework | | SMD vias with soldermask | Probe can't make contact | False failures at ICT, escapes | | No test point on power rail | Can't verify regulation voltage | Power faults go undetected | | Missing ground reference near analog TP | Floating measurement | Noise, false limits | | No reset test point | Can't force DUT into known state | Functional test must rely on POR only | | 0.5 mm or smaller test point pads | Spring pins miss or skip | Intermittent contact, low fixture yield | | Test points at arbitrary coordinates | Off-grid pins, fixture complexity | Higher fixture build cost | ## Boundary Scan (JTAG) Considerations JTAG boundary scan (IEEE 1149.1) tests interconnect and basic component function without a bed-of-nails fixture. For JTAG to work in manufacturing: - All JTAG-capable devices must be in the scan chain (no unconnected TAP pins) - TDO of each device connects to TDI of the next - TMS and TCK are bussed across all devices - TRST is pulled high with a 10k resistor (active-low) - The chain must be accessible through a header or test points Include a populated 10-pin ARM JTAG or 20-pin JTAG header on EVT boards. Depopulate the connector on production builds but keep the footprint and test points. ## Programming Headers and Debug Interfaces | Interface | Header | Signals needed | |-----------|--------|----------------| | ARM SWD | 10-pin Cortex Debug | SWDIO, SWDCLK, GND, VCC, nRESET | | JTAG (ARM) | 20-pin JTAG | TDI, TDO, TMS, TCK, nRESET, VCC, GND | | UART bootloader | 4-pin 2.54 mm | TX, RX, GND, VCC | | SPI flash | 6-pin | CS, CLK, MOSI, MISO, GND, VCC | | FPGA JTAG | 6-pin | TDI, TDO, TMS, TCK, GND, VCC | Many microcontrollers have BOOT0/BOOT1 pins that select the programming mode. These need to be controllable from the test fixture. Add a test point to BOOT0. ## How DFT Enables Automated Testing with OpenHTF and TofuPilot With test points, JTAG access, and a controllable programming interface, you can build a fixture that drives the DUT through a full test sequence without human intervention. ```python filename="tests/pcba_functional_test.py" import time import subprocess import serial import openhtf as htf from tofupilot.openhtf import TofuPilot def program_firmware(test): """Flash firmware over SWD using pyocd.""" result = subprocess.run( ["pyocd", "flash", "--target", "stm32g431rb", "firmware.hex"], capture_output=True, text=True, timeout=30, ) test.logger.info(result.stdout) if result.returncode != 0: test.logger.error(result.stderr) return htf.PhaseResult.STOP @htf.measures( htf.Measurement("vcc_3v3") .in_range(minimum=3250, maximum=3350), htf.Measurement("vbat") .in_range(minimum=3500, maximum=4200), ) def measure_power_rails(test): """Probe VCC and VBAT test points with DMM.""" test.measurements.vcc_3v3 = 3312 # Replace with real DMM read * 1000 test.measurements.vbat = 3850 # Replace with real DMM read * 1000 @htf.measures( htf.Measurement("uart_echo_ok").equals(True), htf.Measurement("uart_response_time") .in_range(minimum=0, maximum=100), ) def test_uart_comms(test): """Send command over UART, verify echo and response time.""" with serial.Serial("/dev/ttyUSB0", baudrate=115200, timeout=1) as port: port.write(b"PING\r\n") t0 = time.monotonic() response = port.readline().decode().strip() elapsed_ms = (time.monotonic() - t0) * 1000 test.measurements.uart_echo_ok = response == "PONG" test.measurements.uart_response_time = round(elapsed_ms, 1) def main(): test = htf.Test( program_firmware, measure_power_rails, test_uart_comms, test_name="PCBA Functional Test", ) with TofuPilot(test): test.execute(test_start=lambda: input("Serial: ")) if __name__ == "__main__": main() ``` Each phase maps to a test point or interface on the board. `program_firmware` uses SWD. `measure_power_rails` uses test points on VCC and VBAT. `test_uart_comms` uses the UART header. ### Well-Designed Board vs. Poorly-Designed Board | Step | Well-designed DUT | Poorly-designed DUT | |------|------------------|---------------------| | Fixture contact | 100% spring pin contact on 100-mil grid | 40% contact due to off-grid and covered vias | | Firmware programming | SWD header, automated with pyocd | Manual USB cable plug, operator-dependent | | Power rail verification | Test point on each rail, DMM probe | No test points, inferred from UART only | | UART comms test | Header with TX/RX/GND, 3-second phase | Requires board modification or probing SMD pad | | Cycle time | 45 seconds automated, zero operator steps | 4 minutes with two operator interventions | | Escape rate | Low (full coverage) | High (partial coverage, manual steps) | ## DFT Checklist | Category | Check | Pass criteria | |----------|-------|---------------| | Test points | All power rails have test points | One TP per rail, top side | | Test points | All ground domains have a ground reference TP | Within 25 mm of critical measurement points | | Test points | Key digital I/O are testable | UART, SPI, I2C, GPIO have test points | | Test points | All TPs are 0.9 mm minimum diameter | Verified in DRC | | Test points | All TPs on 100-mil grid | Verify in layout with grid snap | | Test points | No TPs under connectors, heatsinks, or shields | Manual check | | Vias | No soldermask-covered vias on critical nets | Review with fab DFM report | | JTAG | Scan chain connected end-to-end | TDO_n to TDI_(n+1) | | JTAG | TRST pulled high at power-on | 10k resistor to VCC | | JTAG | JTAG header footprint present | Populated on EVT | | Programming | SWD or JTAG header accessible on top side | Verified with fixture clearance | | Programming | BOOT mode pins accessible as test points | BOOT0 or equivalent exposed | | Debug | UART or SWO accessible via header or TP | Baud rate documented | | Mechanical | Board edge clearance >= 3 mm for all TPs | DRC clean | | Mechanical | Component clearance >= 0.5 mm around TPs | Spring pin travel verified | ### Log HIL Results for Regression Analysis URL: https://www.tofupilot.com/guides/how-to-log-hil-test-results-to-tofupilot-for-regression-analysis Structure HIL test results with firmware versions and environmental data for regression tracking in TofuPilot. HIL tests generate a lot of data: measurements, waveform captures, firmware versions, environmental conditions. Without structured logging, spotting regressions across firmware releases turns into a spreadsheet archaeology project. This guide shows how to tag, attach, and query HIL results in TofuPilot so regressions surface automatically. ## Prerequisites - Python 3.8+ - OpenHTF installed (`pip install openhtf`) - TofuPilot Python client (`pip install tofupilot`) - A working HIL test (see [How to Set Up HIL Testing for Embedded Systems with TofuPilot](/guides/how-to-set-up-hil-testing-for-embedded-systems-with-python-and-tofupilot)) ## Step 1: Tag Runs with Firmware Version and Hardware Revision Every HIL test run should carry the firmware version and hardware revision of the DUT. TofuPilot stores these as structured metadata you can filter and query later. Query the firmware version from the DUT at the start of the test, then pass it to TofuPilot through measurements. ```python filename="hil_regression/main.py" import openhtf as htf from tofupilot.openhtf import TofuPilot from hil_regression.phases import ( read_firmware_info, test_adc_accuracy, test_pwm_output, test_sleep_current, ) def main(): test = htf.Test( read_firmware_info, test_adc_accuracy, test_pwm_output, test_sleep_current, ) with TofuPilot(test): test.execute(test_start=lambda: input("Scan DUT serial number: ")) if __name__ == "__main__": main() ``` The firmware version and hardware revision get recorded as measurements, making them searchable across all runs. ```python filename="hil_regression/phases.py" import openhtf as htf from hil_regression.serial_plug import SerialCommandPlug @htf.plug(serial=SerialCommandPlug) @htf.measures( htf.Measurement("firmware_version"), htf.Measurement("hardware_revision"), htf.Measurement("bootloader_version"), ) def read_firmware_info(test, serial): """Record firmware and hardware identifiers for traceability.""" test.measurements.firmware_version = serial.send_command("VERSION") test.measurements.hardware_revision = serial.send_command("HWREV") test.measurements.bootloader_version = serial.send_command("BLVER") ``` This gives you three filterable fields on every run. When FPY drops after a firmware update, filter by `firmware_version` in TofuPilot's dashboard to isolate the change. ## Step 2: Attach Waveform and Log Files HIL tests often capture oscilloscope waveforms, logic analyzer traces, or DUT console logs. Attach these to the test run so they're available for post-mortem analysis without digging through file shares. ```python filename="hil_regression/capture.py" import csv import time import openhtf as htf from openhtf.plugs import BasePlug class WaveformCapturePlug(BasePlug): """Captures analog samples and saves to CSV for attachment.""" def setUp(self): pass def capture_waveform(self, adc, channel: int, duration_s: float, sample_rate_hz: int) -> str: """Sample an ADC channel and write results to a CSV file.""" filepath = f"/tmp/waveform_ch{channel}_{int(time.time())}.csv" samples = [] interval = 1.0 / sample_rate_hz for i in range(int(duration_s * sample_rate_hz)): voltage = adc.read_voltage(channel) samples.append((i * interval, voltage)) time.sleep(interval) with open(filepath, "w", newline="") as f: writer = csv.writer(f) writer.writerow(["time_s", "voltage_v"]) writer.writerows(samples) return filepath def tearDown(self): pass ``` Then attach the file in your test phase: ```python filename="hil_regression/phases.py" import openhtf as htf from openhtf.util import units from hil_regression.capture import WaveformCapturePlug from hil_regression.analog_plug import AnalogInputPlug from hil_regression.serial_plug import SerialCommandPlug @htf.plug(serial=SerialCommandPlug, adc=AnalogInputPlug, capture=WaveformCapturePlug) @htf.measures( htf.Measurement("pwm_voltage_mean").in_range(minimum=1.6, maximum=1.7).with_units(units.VOLT), ) def test_pwm_output(test, serial, adc, capture): """Command 50% PWM and capture the output waveform.""" serial.send_command("PWM SET 50") # Capture 1 second of data at 1 kHz waveform_path = capture.capture_waveform(adc, channel=3, duration_s=1.0, sample_rate_hz=1000) # Attach the waveform CSV to the test run test.attach("pwm_waveform", waveform_path, "text/csv") # Also record the mean voltage as a measurement import csv with open(waveform_path) as f: reader = csv.DictReader(f) voltages = [float(row["voltage_v"]) for row in reader] test.measurements.pwm_voltage_mean = sum(voltages) / len(voltages) ``` Attachments show up in TofuPilot's run detail view. You can download them later for offline analysis or comparison across firmware versions. ## Step 3: Use Sub-Units for Multi-Board HIL Setups Many products contain multiple boards (main CPU board, power board, sensor board) that get tested together in a HIL fixture. TofuPilot's sub-units let you track each board independently while keeping them linked to the parent assembly. ```python filename="hil_regression/multi_board.py" import openhtf as htf from tofupilot.openhtf import TofuPilot from hil_regression.phases import ( test_cpu_board, test_power_board, test_sensor_board, ) def main(): test = htf.Test( test_cpu_board, test_power_board, test_sensor_board, ) with TofuPilot( test, sub_units=[ {"serial_number": "CPU-BRD-0042", "part_number": "PCB-CPU-R3"}, {"serial_number": "PWR-BRD-0019", "part_number": "PCB-PWR-R2"}, {"serial_number": "SNS-BRD-0088", "part_number": "PCB-SNS-R1"}, ], ): test.execute(test_start=lambda: input("Scan assembly serial number: ")) if __name__ == "__main__": main() ``` Each sub-unit gets its own traceability record in TofuPilot. If the sensor board starts failing after a hardware revision change, you can filter by `PCB-SNS-R1` and see exactly when failures started. ```python filename="hil_regression/board_phases.py" import openhtf as htf from openhtf.util import units from hil_regression.analog_plug import AnalogInputPlug from hil_regression.serial_plug import SerialCommandPlug @htf.plug(serial=SerialCommandPlug) @htf.measures( htf.Measurement("cpu_clock_mhz").in_range(minimum=79, maximum=81).with_units(units.HERTZ), htf.Measurement("cpu_temp").in_range(maximum=85).with_units(units.DEGREE_CELSIUS), ) def test_cpu_board(test, serial): """Verify CPU board clock and temperature.""" test.measurements.cpu_clock_mhz = float(serial.send_command("CLOCK?")) test.measurements.cpu_temp = float(serial.send_command("TEMP?")) @htf.plug(adc=AnalogInputPlug) @htf.measures( htf.Measurement("pwr_12v_rail").in_range(minimum=11.4, maximum=12.6).with_units(units.VOLT), htf.Measurement("pwr_efficiency_pct").in_range(minimum=85), ) def test_power_board(test, adc): """Verify power board output rails and efficiency.""" test.measurements.pwr_12v_rail = adc.read_voltage(channel=0) vin = adc.read_voltage(channel=4) vout = adc.read_voltage(channel=5) test.measurements.pwr_efficiency_pct = (vout / vin) * 100 if vin > 0 else 0 @htf.plug(serial=SerialCommandPlug, adc=AnalogInputPlug) @htf.measures( htf.Measurement("sensor_offset_mv").in_range(minimum=-5, maximum=5), htf.Measurement("sensor_gain_error_pct").in_range(minimum=-1, maximum=1), ) def test_sensor_board(test, serial, adc): """Verify sensor board calibration.""" serial.send_command("SENSOR CAL_CHECK") test.measurements.sensor_offset_mv = float(serial.send_command("SENSOR OFFSET?")) test.measurements.sensor_gain_error_pct = float(serial.send_command("SENSOR GAIN_ERR?")) ``` ## Detecting Regressions in TofuPilot TofuPilot tracks every measurement value across all runs. To detect firmware regressions: 1. **Filter by firmware version.** Open the procedure's Analytics tab and filter runs by the `firmware_version` measurement. Compare pass rates and measurement distributions between versions. 2. **Check measurement trends.** TofuPilot's trend charts show measurement values over time. A sudden shift after a firmware update is a clear regression signal. 3. **Compare Cpk by version.** If a measurement's Cpk drops after a firmware change, the process capability has degraded. ## Manual Tracking vs TofuPilot | Aspect | Spreadsheet / Manual | TofuPilot | |--------|---------------------|-----------| | Firmware version tagging | Copy-paste into a column | Automatic per-run metadata | | Waveform storage | Shared drive with naming conventions | Attached to the run, always findable | | Multi-board traceability | Separate sheets or tabs | Sub-units linked to parent assembly | | Regression detection | Manual chart inspection | Filter by firmware version, compare trends | | Cross-station comparison | Merge files from different PCs | All stations upload to one workspace | | Historical lookup | "Which folder was that in?" | Search by serial number, part number, or date | | Audit trail | Hope nobody deleted a row | Immutable records with timestamps | ### HIL Testing for Embedded Systems URL: https://www.tofupilot.com/guides/how-to-set-up-hil-testing-for-embedded-systems-with-python-and-tofupilot Set up hardware-in-the-loop testing for embedded systems using Python, GPIO control, and TofuPilot logging for automated regression analysis. Hardware-in-the-loop (HIL) testing lets you verify embedded firmware against real hardware without manually poking at it with a multimeter. This guide walks through building a Python HIL test that controls GPIOs, reads analog signals, talks to the DUT over serial, and logs everything to TofuPilot. ## Prerequisites - Python 3.8+ - OpenHTF installed (`pip install openhtf`) - TofuPilot Python client (`pip install tofupilot`) - A DUT with GPIO, ADC, and UART interfaces - A test fixture (Raspberry Pi, NI DAQ, or similar IO hardware) - PySerial (`pip install pyserial`) ## What Is HIL Testing? In a HIL setup, your DUT connects to a test fixture that simulates the real-world signals it would see in production. The fixture generates stimuli (voltage levels, digital pulses, serial commands) and measures the DUT's responses. Your test script orchestrates the whole sequence. A typical HIL bench looks like this: - **Test host** (PC or Raspberry Pi) runs the test script - **Digital IO** (GPIO pins or a digital IO card) toggles inputs and reads outputs on the DUT - **Analog IO** (DAQ or ADC/DAC) generates analog stimuli and measures DUT outputs - **Serial link** (UART, SPI, I2C) sends commands and reads status from the DUT firmware The test script drives all three channels, checks the DUT's behavior at each step, and records pass/fail results. ## Step 1: Control Digital IO with GPIO Most HIL setups need to toggle DUT inputs and read DUT outputs. If you're using a Raspberry Pi as your test host, RPi.GPIO works fine. For a PC-based setup, you'd swap this for your IO card's SDK. ```python filename="hil_test/gpio_plug.py" import openhtf as htf from openhtf.plugs import BasePlug try: import RPi.GPIO as GPIO except ImportError: GPIO = None # Allows development on non-Pi machines class GpioPowerPlug(BasePlug): """Controls DUT power and reads digital status pins.""" POWER_PIN = 17 STATUS_PIN = 27 def setUp(self): if GPIO is None: raise RuntimeError("RPi.GPIO not available on this platform") GPIO.setmode(GPIO.BCM) GPIO.setup(self.POWER_PIN, GPIO.OUT, initial=GPIO.LOW) GPIO.setup(self.STATUS_PIN, GPIO.IN, pull_up_down=GPIO.PUD_DOWN) def power_on(self): GPIO.output(self.POWER_PIN, GPIO.HIGH) def power_off(self): GPIO.output(self.POWER_PIN, GPIO.LOW) def read_status(self): return GPIO.input(self.STATUS_PIN) def tearDown(self): GPIO.output(self.POWER_PIN, GPIO.LOW) GPIO.cleanup([self.POWER_PIN, self.STATUS_PIN]) ``` The plug handles setup and teardown automatically. OpenHTF calls `setUp()` before the first phase that uses it and `tearDown()` after the test finishes, so your DUT always gets powered off cleanly. ## Step 2: Read Analog Signals with a DAQ For analog measurements, you'll need an ADC or DAQ. This example uses an MCC DAQ (via the `uldaq` library), but the pattern works the same with NI-DAQmx, Labjack, or an ADS1115 on I2C. ```python filename="hil_test/analog_plug.py" from openhtf.plugs import BasePlug from uldaq import DaqDevice, InterfaceType, AiInputMode, Range class AnalogInputPlug(BasePlug): """Reads analog voltages from a USB DAQ.""" def setUp(self): devices = DaqDevice.get_inventory(InterfaceType.USB) if not devices: raise RuntimeError("No DAQ device found") self.daq = DaqDevice(devices[0]) self.daq.connect() self.ai = self.daq.get_ai_device() def read_voltage(self, channel: int) -> float: """Read a single-ended voltage from the specified channel.""" return self.ai.a_in(channel, AiInputMode.SINGLE_ENDED, Range.BIP10VOLTS, 0) def tearDown(self): self.daq.disconnect() self.daq.release() ``` ## Step 3: Communicate with the DUT over Serial Most embedded targets expose a UART debug console or command interface. Wrap it in a plug so OpenHTF manages the connection lifecycle. ```python filename="hil_test/serial_plug.py" import time import serial from openhtf.plugs import BasePlug class SerialCommandPlug(BasePlug): """Sends commands to the DUT over UART and reads responses.""" PORT = "/dev/ttyUSB0" BAUDRATE = 115200 TIMEOUT = 2.0 def setUp(self): self.ser = serial.Serial(self.PORT, self.BAUDRATE, timeout=self.TIMEOUT) time.sleep(0.5) # Wait for DUT bootloader self.ser.reset_input_buffer() def send_command(self, cmd: str) -> str: """Send a command and return the response line.""" self.ser.write(f"{cmd}\r\n".encode()) return self.ser.readline().decode().strip() def get_firmware_version(self) -> str: return self.send_command("VERSION") def tearDown(self): self.ser.close() ``` ## Step 4: Write OpenHTF Phases Each test phase exercises one aspect of the DUT. Plug injection uses the `@htf.plug` decorator. ```python filename="hil_test/phases.py" import time import openhtf as htf from openhtf.util import units from hil_test.gpio_plug import GpioPowerPlug from hil_test.analog_plug import AnalogInputPlug from hil_test.serial_plug import SerialCommandPlug @htf.plug(gpio=GpioPowerPlug) @htf.measures( htf.Measurement("boot_status").equals(1) ) def power_on_and_check_boot(test, gpio): """Power on the DUT and verify it boots.""" gpio.power_on() time.sleep(2.0) # Wait for boot test.measurements.boot_status = gpio.read_status() @htf.plug(serial=SerialCommandPlug) @htf.measures( htf.Measurement("firmware_version") ) def read_firmware_version(test, serial): """Query the DUT firmware version over UART.""" version = serial.get_firmware_version() test.measurements.firmware_version = version @htf.plug(gpio=GpioPowerPlug, adc=AnalogInputPlug) @htf.measures( htf.Measurement("vout_3v3").in_range(minimum=3.1, maximum=3.5).with_units(units.VOLT), htf.Measurement("vout_5v0").in_range(minimum=4.75, maximum=5.25).with_units(units.VOLT), ) def measure_power_rails(test, gpio, adc): """Verify the DUT power rails are within spec.""" test.measurements.vout_3v3 = adc.read_voltage(channel=0) test.measurements.vout_5v0 = adc.read_voltage(channel=1) @htf.plug(serial=SerialCommandPlug, adc=AnalogInputPlug) @htf.measures( htf.Measurement("dac_output_1v0").in_range(minimum=0.95, maximum=1.05).with_units(units.VOLT), ) def test_dac_output(test, serial, adc): """Command the DUT to output 1.0V on its DAC, then measure it.""" serial.send_command("DAC SET 1000") # 1000 mV time.sleep(0.5) test.measurements.dac_output_1v0 = adc.read_voltage(channel=2) @htf.plug(gpio=GpioPowerPlug) def power_off(test, gpio): """Shut down the DUT.""" gpio.power_off() ``` ## Step 5: Integrate with TofuPilot Wrap the OpenHTF test execution in `TofuPilot` to automatically log results, measurements, and DUT metadata. ```python filename="hil_test/main.py" import openhtf as htf from tofupilot.openhtf import TofuPilot from hil_test.phases import ( power_on_and_check_boot, read_firmware_version, measure_power_rails, test_dac_output, power_off, ) test = htf.Test( power_on_and_check_boot, read_firmware_version, measure_power_rails, test_dac_output, power_off, ) with TofuPilot(test): test.execute(test_start=lambda: input("Scan DUT serial number: ")) ``` Every test run uploads to TofuPilot with the DUT serial number, all measurements and limits, and pass/fail status per phase. ## Troubleshooting | Symptom | Likely Cause | Fix | |---------|-------------|-----| | `RuntimeError: RPi.GPIO not available` | Running on a non-Pi machine | Use a GPIO simulator or deploy to the Pi | | `serial.SerialException: could not open port` | Wrong port or DUT not powered | Check `PORT` constant, verify DUT has power | | `No DAQ device found` | DAQ not connected or driver missing | Run `lsusb` to verify, install `uldaq` drivers | | ADC reads 0V on all channels | Wrong input mode or range | Confirm `SINGLE_ENDED` vs `DIFFERENTIAL`, check wiring | | DUT doesn't respond to serial commands | Baud rate mismatch or wrong line ending | Match DUT firmware settings, try `\r\n` vs `\n` | | Measurements intermittently fail | Settling time too short after stimulus | Increase `time.sleep()` after state changes | ### Track Serial Numbers and Sub-Assemblies URL: https://www.tofupilot.com/guides/how-to-track-serial-numbers-and-sub-assemblies-with-tofupilot Record parent/child serial number relationships in your test data to trace sub-assemblies across boards, modules, and final products. Most hardware products contain sub-assemblies, each with its own serial number. TofuPilot's `sub_units` feature lets you record which components went into which parent unit, giving you full BOM traceability from a single test script. ## Why Sub-Assembly Tracking Matters A finished product might contain a power supply board, a compute module, and a sensor array. Each has its own serial number and its own test history. When a field failure points to a specific component, you need to know which parent units contain that component. This is standard practice for IPC-1782 traceability and required in regulated industries. TofuPilot links parent and child serial numbers automatically when you include them in your test runs. ## Recording Sub-Unit Serial Numbers Use the `sub_units` parameter in TofuPilot to declare which components were installed in the DUT during assembly. ```python filename="test_assembly.py" # Record sub-assembly serial numbers during final assembly test import openhtf as htf from tofupilot.openhtf import TofuPilot @htf.measures( htf.Measurement("system_power_on").in_range(minimum=1, maximum=1), htf.Measurement("communication_check").in_range(minimum=1, maximum=1), ) def system_integration_test(test): test.measurements.system_power_on = 1 test.measurements.communication_check = 1 def main(): test = htf.Test(system_integration_test) with TofuPilot( test, sub_units=[ {"serial_number": "PSU-2026-0441"}, {"serial_number": "CPU-2026-1187"}, {"serial_number": "SNS-2026-0893"}, ], ): test.execute(test_start=lambda: "ASSY-2026-0072") if __name__ == "__main__": main() ``` After this test uploads, TofuPilot's unit page for `ASSY-2026-0072` shows three linked sub-assemblies. Click any sub-unit serial to see its own test history. ## Scanning Sub-Unit Serials During Assembly In practice, operators scan component serial numbers as they install them. You can collect these in an OpenHTF phase and pass them to TofuPilot. ```python filename="test_scan_subunits.py" # Operator scans sub-assembly serials during assembly import openhtf as htf from openhtf.util import units from tofupilot.openhtf import TofuPilot # Store scanned serials at module level scanned_sub_units = [] @htf.measures( htf.Measurement("psu_serial"), htf.Measurement("cpu_serial"), ) def scan_components(test): psu = input("Scan PSU serial: ") cpu = input("Scan CPU serial: ") test.measurements.psu_serial = psu test.measurements.cpu_serial = cpu scanned_sub_units.append({"serial_number": psu}) scanned_sub_units.append({"serial_number": cpu}) @htf.measures( htf.Measurement("power_rail_5v").in_range(minimum=4.8, maximum=5.2).with_units(units.VOLT), htf.Measurement("power_rail_3v3").in_range(minimum=3.1, maximum=3.5).with_units(units.VOLT), ) def power_validation(test): test.measurements.power_rail_5v = 5.02 test.measurements.power_rail_3v3 = 3.31 def main(): test = htf.Test(scan_components, power_validation) with TofuPilot(test, sub_units=scanned_sub_units): test.execute(test_start=lambda: input("Scan assembly serial: ")) if __name__ == "__main__": main() ``` ## Multi-Level Assemblies For products with nested assemblies (a module inside a board inside a chassis), test each level separately with its own sub-units. TofuPilot builds the hierarchy automatically. ```python filename="test_nested_assembly.py" # Test a module, then test the board that contains it import openhtf as htf from tofupilot.openhtf import TofuPilot @htf.measures( htf.Measurement("module_self_test").in_range(minimum=1, maximum=1), ) def module_test(test): test.measurements.module_self_test = 1 @htf.measures( htf.Measurement("board_communication").in_range(minimum=1, maximum=1), ) def board_test(test): test.measurements.board_communication = 1 def main(): # First: test the module by itself module_serial = "MOD-2026-0551" t1 = htf.Test(module_test) with TofuPilot(t1): t1.execute(test_start=lambda: module_serial) # Second: test the board, declaring the module as a sub-unit board_serial = "BRD-2026-0112" t2 = htf.Test(board_test) with TofuPilot(t2, sub_units=[{"serial_number": module_serial}]): t2.execute(test_start=lambda: board_serial) if __name__ == "__main__": main() ``` In TofuPilot's dashboard, searching for the board serial shows the module as a sub-unit. Searching for the module serial shows its own test results plus the parent board it was installed in. ## BOM Traceability Use Cases Once sub-assembly links exist in TofuPilot, you can answer questions that matter in production: - **Field failure investigation.** A sensor module fails in the field. Search its serial number to find every parent unit that contains the same module type from the same production batch. - **Component recall.** A supplier flags a batch of capacitors. If your test phases record component lot numbers as measurements, you can trace which assemblies used them. - **Yield by component.** TofuPilot's dashboard shows FPY broken down by any dimension. If a specific sub-assembly batch is causing failures, you'll see it in the data. ## Viewing Sub-Assembly Links Open any unit's page in TofuPilot to see its sub-assembly tree. The page shows: - Direct child components with their serial numbers - Each child's own test history (click through to view) - The parent assembly, if this unit is itself a sub-component - All test runs for the unit, across every production stage This gives quality engineers and auditors a single place to trace any component through the entire product hierarchy. ### Monitor Test Stations Across Factories URL: https://www.tofupilot.com/guides/how-to-monitor-test-stations-across-factories-with-tofupilot Track test station performance across multiple factories from a single dashboard. Compare yield, throughput, and measurement drift by site and station. When the same test procedure runs on dozens of stations across multiple factories, you need a single place to see what's happening everywhere. TofuPilot gives you cross-factory visibility without building custom dashboards or aggregating CSVs. ## Why Multi-Site Monitoring Matters A test that passes 98% at your Shenzhen factory and 94% at your Guadalajara factory tells you something is wrong. The problem could be equipment calibration, operator training, environmental conditions, or component lot variation. You can't fix what you can't see. Station-level data also helps you spot individual machines drifting before they start producing false passes or unnecessary failures. ## Tag Each Station with Metadata OpenHTF lets you attach metadata to every test run. Use `station_id` to identify the physical station and add factory or site information so TofuPilot can group results. ```python filename="test_board.py" import openhtf as htf from openhtf.util import units from tofupilot.openhtf import TofuPilot @htf.measures( htf.Measurement("supply_voltage") .in_range(minimum=4.9, maximum=5.1) .with_units(units.VOLT) ) def test_supply_voltage(test): voltage = 5.03 # Read from instrument test.measurements.supply_voltage = voltage def main(): test = htf.Test( test_supply_voltage, station_id="SMT-LINE3-ICT-02", ) with TofuPilot(test): test.execute(test_start=lambda: "PCB-2026-00451") if __name__ == "__main__": main() ``` The `station_id` value should be unique and descriptive. A good convention is `{SITE}-{LINE}-{FUNCTION}-{NUMBER}`, like `SZ-L3-ICT-02` for Shenzhen, Line 3, In-Circuit Test, Station 2. ## Standardize Station Naming Across Sites Consistent naming is critical. If one factory uses `ICT_01` and another uses `ict-station-1`, you can't filter or compare them reliably. Define a naming convention before deployment: | Field | Format | Example | |-------|--------|---------| | Site code | 2-3 letter abbreviation | `SZ`, `GDL`, `AUS` | | Line number | `L` + number | `L1`, `L3` | | Test type | Standard abbreviation | `ICT`, `FCT`, `EOL` | | Station number | Zero-padded number | `01`, `02`, `12` | This gives you station IDs like `SZ-L3-ICT-02` or `GDL-L1-FCT-05` that are readable and sortable. ## Run the Same Procedure Everywhere Multi-site monitoring only works if every factory runs the same test procedure. Version-control your test scripts in Git and deploy them to all sites from a single repository. ```python filename="test_final.py" import openhtf as htf from openhtf.util import units from tofupilot.openhtf import TofuPilot @htf.measures( htf.Measurement("output_current") .in_range(minimum=0.95, maximum=1.05) .with_units(units.AMPERE), htf.Measurement("firmware_checksum") .equals("a3f8c2d1") ) def test_output_and_firmware(test): test.measurements.output_current = 1.01 # Read from load test.measurements.firmware_checksum = "a3f8c2d1" # Read from DUT def main(): test = htf.Test( test_output_and_firmware, station_id="AUS-L2-FCT-01", ) with TofuPilot(test): test.execute(test_start=lambda: "UNIT-88210") if __name__ == "__main__": main() ``` When every station uploads results from the same procedure, TofuPilot automatically groups them. You can then filter by station, site, or time range to compare performance. ## View Cross-Factory Data in TofuPilot Once runs are uploading from multiple sites, TofuPilot's station dashboard shows: - **Per-station yield** so you can spot underperforming machines instantly - **Measurement distributions by station** to catch calibration drift before it causes failures - **Throughput by station** to identify bottlenecks and downtime - **Failure Pareto by site** to see whether failure modes differ between factories Filter by station ID prefix (like `SZ-` or `GDL-`) to compare entire factories side by side. Drill into individual stations to investigate anomalies. ## Handle Station Offline Detection A station that stops reporting is a problem. It might be down for maintenance, or it might be running tests that aren't uploading. TofuPilot tracks when each station last reported, so gaps in data are visible immediately. Build a habit of checking the station overview at the start of each shift. If a station hasn't reported in the expected window, investigate before production continues. ### What Is Process Capability (Cp and Cpk) URL: https://www.tofupilot.com/guides/what-is-process-capability-cp-and-cpk-with-tofupilot Understand Cp and Cpk formulas, what different Cpk values mean for your process, and how TofuPilot calculates them from your test data. Process capability tells you whether your manufacturing process can consistently produce units within specification limits. Cp measures the potential, Cpk measures the reality. Together they reveal whether your process is capable and centered. ## The Formulas **Cp** measures how much of the specification window your process variation uses: **Cp = (USL - LSL) / 6σ** Where USL is the upper specification limit, LSL is the lower specification limit, and σ is the process standard deviation. Cp only looks at spread. It ignores whether your process is centered between the limits. That's where Cpk comes in. **Cpk = min((USL - μ) / 3σ, (μ - LSL) / 3σ)** Where μ is the process mean. Cpk takes the worse side. If your process drifts toward one limit, Cpk drops even if Cp stays high. ## What Cpk Values Mean | Cpk Value | Interpretation | Defect Rate (approx.) | |-----------|---------------|----------------------| | < 1.0 | Process is not capable. Units are falling outside limits. | > 2,700 ppm | | 1.0 | Barely capable. The 3-sigma edge touches the spec limit. | ~2,700 ppm | | 1.0 - 1.33 | Marginally capable. Acceptable for non-critical dimensions. | 2,700 - 63 ppm | | 1.33 | Commonly accepted minimum for production processes. | ~63 ppm | | 1.33 - 1.67 | Good capability. Reasonable margin for process drift. | 63 - 0.6 ppm | | > 1.67 | Excellent. Process has significant margin within spec. | < 0.6 ppm | A practical way to think about it: Cpk 1.33 means your process uses 75% of the specification window, leaving 25% as buffer for drift and variation. Most automotive and aerospace standards require Cpk >= 1.33 for production, with 1.67 for safety-critical parameters. ## The Relationship Between Cp and Cpk When Cp equals Cpk, your process is perfectly centered between the specification limits. When Cpk is significantly lower than Cp, your process has drifted off-center. | Scenario | Cp | Cpk | Diagnosis | |----------|-----|------|-----------| | Centered and capable | 1.5 | 1.5 | Ideal state | | Off-center but capable spread | 1.5 | 0.9 | Process mean has drifted. Re-center it. | | Wide variation, centered | 0.8 | 0.8 | Too much variation. Reduce σ. | | Wide variation, off-center | 0.8 | 0.4 | Both problems. Fix variation first. | This distinction matters for corrective action. If Cp is fine but Cpk is low, you need to adjust the process mean (a calibration issue). If Cp itself is low, you need to reduce variation (a fundamental process issue). ## Feeding Measurement Data to TofuPilot For TofuPilot to calculate Cpk, your test code needs to define measurements with upper and lower limits. Here's an OpenHTF test that measures critical parameters on a sensor module: ```python filename="test_sensor_calibration.py" # Sensor calibration test with limits for Cpk analysis import openhtf as htf from openhtf.util import units from tofupilot.openhtf import TofuPilot @htf.measures( htf.Measurement("output_voltage") .in_range(minimum=2.45, maximum=2.55) .with_units(units.VOLT), htf.Measurement("offset_voltage") .in_range(minimum=-5.0, maximum=5.0), htf.Measurement("sensitivity") .in_range(minimum=195, maximum=205), htf.Measurement("noise_floor") .in_range(maximum=500), htf.Measurement("response_time") .in_range(maximum=10.0), ) def sensor_validation(test): test.measurements.output_voltage = 2.501 test.measurements.offset_voltage = 0.8 test.measurements.sensitivity = 200.3 test.measurements.noise_floor = 320 test.measurements.response_time = 4.7 def main(): test = htf.Test(sensor_validation) with TofuPilot(test): test.execute(test_start=lambda: "SENSOR-2024-1587") if __name__ == "__main__": main() ``` The `in_range()` calls define USL and LSL. When you set `minimum` and `maximum`, TofuPilot gets both limits and can compute Cp and Cpk. When you set only `maximum`, TofuPilot computes a one-sided capability index. ## Reading Cpk in TofuPilot Once enough test runs accumulate (typically 30+ data points for statistical significance), TofuPilot's measurement analytics show you: - **Cpk value per measurement** calculated from your production data. You can see at a glance which measurements are capable and which aren't. - **Measurement histograms** overlaid with specification limits, so you can visually assess distribution shape, centering, and spread. - **Control charts** that track measurement values over time, revealing drift before it causes Cpk to drop below your threshold. You don't need to export data and calculate Cpk in a spreadsheet. Define your limits in OpenHTF, run your tests, and TofuPilot computes the rest. ## Common Pitfalls **Not enough data.** Cpk calculated from 10 units is unreliable. Wait for at least 30 data points, ideally 50+, before making process decisions based on Cpk. **Confusing specification limits with control limits.** Specification limits (USL/LSL) come from your product requirements. Control limits come from your process data. Cpk uses specification limits. **Ignoring non-normal distributions.** The standard Cpk formula assumes a normal distribution. If your measurement data is skewed (common with one-sided specs like noise floor), the standard formula may overestimate capability. **Mixing populations.** If you calculate Cpk across data from two different test stations with different calibrations, the combined σ will be inflated. Check per-station Cpk first. ### Python vs LabVIEW for Manufacturing Test URL: https://www.tofupilot.com/guides/python-vs-labview-for-manufacturing-test-a-practical-comparison A structured comparison of Python and LabVIEW for manufacturing test automation, with cost analysis, feature comparison, and real migration considerations. Python and LabVIEW are the two most common languages for manufacturing test. LabVIEW has been the default for 30 years. Python is replacing it. This guide compares them on the things that actually matter for production test: cost, instrument support, version control, deployment, and analytics. ## Head-to-Head Comparison | Feature | Python | LabVIEW | |---------|--------|---------| | **License cost** | Free | $3,160-4,840/seat/year | | **Platform** | Linux, macOS, Windows | Windows only (runtime on Linux/macOS) | | **Language type** | Text-based | Graphical (dataflow) | | **Version control** | Git-native (text files) | Difficult (binary .vi files) | | **Code review** | Standard pull requests | Requires LabVIEW to view | | **Package ecosystem** | 400K+ packages (PyPI) | NI packages + limited community | | **Instrument control** | PyVISA (open source) | NI drivers (proprietary) | | **Test framework** | OpenHTF, pytest | Built-in test tools + TestStand | | **CI/CD** | Native (GitHub Actions, Jenkins, etc.) | Complex to integrate | | **Learning curve** | Days (for programmers) | Weeks (unique paradigm) | | **Hiring pool** | Very large | Small and shrinking | | **Execution speed** | Interpreted (fast enough for test) | Compiled (slightly faster) | | **Data analytics** | TofuPilot, pandas, numpy | TDMS + custom tools | ## Cost Analysis For a team of 10 test engineers running 20 test stations: | Item | Python | LabVIEW | |------|--------|---------| | Development licenses (10) | $0 | $31,600-48,400/year | | Runtime licenses (20 stations) | $0 | Included (but limited without dev license) | | Test framework | $0 (OpenHTF) | $4,310/seat (TestStand) | | Analytics platform | TofuPilot pricing | Custom development cost | | **Year 1 total** | TofuPilot only | $75,000-90,000+ | | **Year 5 total** | TofuPilot only | $375,000-450,000+ | The savings compound. Every new test station with Python costs $0 in licensing. Every new station with LabVIEW adds license fees. ## Instrument Control: PyVISA vs NI Drivers Both languages can control the same instruments. The approach differs. ### Python (PyVISA) ```python filename="comparison/python_instrument.py" import pyvisa rm = pyvisa.ResourceManager("@py") dmm = rm.open_resource("TCPIP::192.168.1.100::INSTR") dmm.timeout = 5000 voltage = float(dmm.query(":MEAS:VOLT:DC?")) print(f"Voltage: {voltage:.4f} V") dmm.close() ``` ### LabVIEW In LabVIEW, you'd use an instrument driver VI: 1. Open VISA session (VISA Open) 2. Write SCPI command (VISA Write) 3. Read response (VISA Read) 4. Parse string to number (Fract/Exp String to Number) 5. Close session (VISA Close) 6. Wire error cluster through everything Five nodes and error wires vs. four lines of Python. The SCPI commands are identical. The instrument doesn't care which language sends them. | Aspect | PyVISA | NI LabVIEW Drivers | |--------|--------|-------------------| | GPIB support | Via NI-VISA backend | Native | | USB-TMC support | Native (pyvisa-py) | Native | | Ethernet/LXI support | Native | Native | | Serial support | Native | Native | | Driver availability | Generic SCPI (works with any vendor) | Vendor-specific instrument drivers | | Custom instruments | Write a Python class | Create a .vi driver | ## Test Data and Analytics This is where Python with TofuPilot has a clear advantage over LabVIEW. | Capability | Python + TofuPilot | LabVIEW + TDMS | |-----------|-------------------|----------------| | Data logging | Automatic (one line) | Manual TDMS write | | FPY tracking | Automatic dashboard | Build it yourself | | Cpk analysis | Automatic per measurement | Build it yourself | | Control charts | Automatic | Build it yourself | | Multi-station view | Automatic aggregation | Manual file aggregation | | Failure Pareto | Automatic | Build it yourself | | API access | REST API | Parse TDMS files | | Historical analysis | Cloud storage, instant queries | TDMS files on file server | With LabVIEW, you write the test and then spend weeks building analytics tools. With Python and TofuPilot, analytics are automatic from the first test run. ## Version Control This is LabVIEW's biggest weakness for team collaboration. | Scenario | Python | LabVIEW | |----------|--------|---------| | View changes | `git diff` in any terminal | Need LabVIEW open + LV Merge tool | | Code review | GitHub/GitLab pull request | Can't see .vi diffs in browser | | Merge conflict | Text-based merge (standard) | Often requires manual rebuild | | Branch strategy | Standard Git flow | Risky (binary merge conflicts) | | Blame/history | `git blame` per line | Per-file only (binary) | | Repo size | Small (text files) | Large (binary VIs with front panels) | ## When to Stay with LabVIEW LabVIEW is still the right choice in some scenarios: - **NI hardware-only environment.** If your test system is entirely NI PXI/cDAQ/FPGA, the LabVIEW integration is unmatched. Python wrappers exist (nidaqmx) but LabVIEW has deeper access to NI hardware features. - **Real-time and FPGA targets.** LabVIEW RT and LabVIEW FPGA have no Python equivalent for NI hardware. If you deploy to NI real-time controllers or FPGAs, you need LabVIEW. - **Large existing codebase with no migration budget.** If it works and you're not adding test stations, the cost of migration may not be justified. - **Non-programmer team.** If your test engineers are purely EE/ME with no programming background, LabVIEW's visual approach may be easier to learn. (Though most engineers learn Python quickly.) ## When to Switch to Python Python is the better choice when: - **You're starting a new test system.** Zero reason to start with LabVIEW licensing costs today. - **You're scaling beyond 5 test stations.** License costs add up fast. - **You need cross-platform.** Python runs on any OS. LabVIEW development is Windows-only. - **You want CI/CD.** Python tests run in any CI pipeline. LabVIEW CI requires NI CLI tools and Windows runners. - **Your team knows Python.** Most EEs learn Python in school. LabVIEW is niche. - **You need analytics.** TofuPilot gives you FPY, Cpk, and control charts out of the box. With LabVIEW, you build everything. ## Migration Strategy Don't rewrite everything at once. Migrate incrementally: 1. **New tests in Python.** Any new product gets a Python test script. 2. **Wrap LabVIEW with Python.** Call existing LabVIEW code from Python using the `subprocess` module or NI's Python API. 3. **Convert high-maintenance tests first.** Tests that change frequently benefit most from Python's version control. 4. **Convert one station at a time.** Validate Python results match LabVIEW results before switching. 5. **Keep LabVIEW for FPGA/RT.** If you need NI real-time or FPGA targets, keep LabVIEW for those specific subsystems. ### Use Control Charts for Tests URL: https://www.tofupilot.com/guides/how-to-use-control-charts-for-tests-with-tofupilot Learn how to use control charts to monitor test process stability, distinguish control limits from spec limits, and detect out-of-control conditions with. Control charts show whether your test process is stable or drifting out of control. They're the foundation of Statistical Process Control (SPC) and let you catch problems before they cause failures. ## What Control Charts Show A control chart plots individual measurements (or subgroup statistics) over time against three lines: a center line (process mean), an Upper Control Limit (UCL), and a Lower Control Limit (LCL). These limits are calculated from your actual data, typically at 3-sigma from the mean. Points inside the control limits mean your process is behaving normally. Points outside, or patterns like seven consecutive points on one side of the center line, signal something has changed. ## Control Limits vs. Spec Limits This distinction matters. Spec limits (USL/LSL) define what's acceptable for the product. Control limits (UCL/LCL) describe what your process actually does. A process can be in control but out of spec (consistently producing bad parts). It can also be in spec but out of control (passing today, but unpredictably). Control charts catch the second case, which spec limits alone miss. ## Writing Tests That Feed Control Charts Good control chart data starts with well-structured measurements. Each measurement needs a name, a unit, and consistent conditions across runs. ```python filename="control_chart_test.py" import openhtf as htf from openhtf.util import units from tofupilot.openhtf import TofuPilot @htf.measures( htf.Measurement("supply_voltage") .with_units(units.VOLT) .in_range(minimum=4.75, maximum=5.25), htf.Measurement("supply_current") .with_units(units.AMPERE) .in_range(minimum=0.090, maximum=0.110), htf.Measurement("output_frequency") .with_units(units.HERTZ) .in_range(minimum=999.5, maximum=1000.5), ) def power_supply_check(test): test.measurements.supply_voltage = 5.02 test.measurements.supply_current = 0.0987 test.measurements.output_frequency = 1000.1 @htf.measures( htf.Measurement("signal_amplitude") .in_range(minimum=-1.0, maximum=1.0), htf.Measurement("signal_noise_floor") .in_range(maximum=-60.0), ) def signal_integrity_check(test): test.measurements.signal_amplitude = 0.3 test.measurements.signal_noise_floor = -72.5 def main(): test = htf.Test(power_supply_check, signal_integrity_check) with TofuPilot(test): test.execute(test_start=lambda: "PCB-001") if __name__ == "__main__": main() ``` Each run uploads measurements to TofuPilot. Over time, these build the dataset that control charts need. ## Choosing the Right Chart Type For individual measurements (one value per run), use an I-MR chart (Individual and Moving Range). This is the most common case in electronics testing where each DUT produces one reading per measurement. For subgrouped data (multiple samples per batch), X-bar and R charts track the subgroup mean and range. This applies when you test multiple units from the same production lot and want to monitor lot-to-lot variation. ## Reading Control Charts in TofuPilot TofuPilot's measurement analytics automatically generates control charts from your test data. Open any measurement's detail view to see the time-series plot with calculated control limits. Watch for these signals: - A single point beyond UCL or LCL - Seven or more consecutive points above or below the center line - Six or more consecutive points trending in one direction - Alternating patterns (up-down-up-down) suggesting measurement system issues When you spot any of these, investigate the root cause before the process produces out-of-spec parts. ## Keeping Charts Useful Recalculate control limits periodically, especially after process changes. If you've fixed a root cause and the process has genuinely improved, update the baseline. Stale limits hide real shifts and generate false alarms. Group your charts by test station. Mixing data from different stations, fixtures, or environments inflates variation and makes the charts less sensitive to real changes on any single station. ### Version Control Hardware Tests with Git URL: https://www.tofupilot.com/guides/how-to-version-control-hardware-tests-with-git-and-tofupilot Learn how to version control hardware test scripts with Git and link test results to code versions in TofuPilot. When a test starts failing on the production floor, the first question is always: "What changed?" If your test scripts aren't version controlled, you can't answer that. Git gives you a complete history of every change to every test. TofuPilot links each test run to the exact code version that produced it. This guide covers Git workflow for test projects, embedding commit hashes in test metadata, and a practical .gitignore for hardware test repos. ## Why Version Control Matters for Hardware Tests Hardware test scripts aren't throwaway code. They're part of your quality system. Regulators, auditors, and your future self need to know: - Which version of the test script produced a given result - Who changed a measurement limit, and when - Whether the test that ran in DVT is the same one running in PVT Without Git, you get folders named `test_v2_final_FINAL_john.py`. With Git, you get a clean audit trail. ## Git Workflow for Test Scripts A simple branching strategy works well for test projects. You don't need GitFlow. You need something your team will actually use. ```python filename="git_workflow.txt" main ──●──●──●──●──●──●── (production-ready tests) \ / feature/add- ──●──●──●──● (new test or change) thermal-test ``` **main** is always deployable to production stations. Every commit on main has been reviewed and tested. **Feature branches** are where you develop new tests or modify existing ones. Name them descriptively: `feature/add-thermal-cycling`, `fix/dmm-timeout-handling`, `update/voltage-limits-rev-c`. Basic workflow: ```python filename="workflow_commands.sh" # Start a new test change git checkout -b feature/add-current-measurement # Make your changes, test locally git add tests/power_rail_fct.py git commit -m "Add idle current measurement to power rail FCT" # Push and create a pull request for review git push -u origin feature/add-current-measurement # After review, merge to main git checkout main git pull git merge feature/add-current-measurement git push ``` Tag releases when you deploy to production stations: ```python filename="tagging.sh" # Tag a release before deploying to production git tag -a v1.2.0 -m "Add thermal cycling test, update voltage limits for Rev C" git push origin v1.2.0 ``` ## Embed Git Commit Hash in Test Metadata Link every test run to the exact code that produced it. This is the key traceability link between your test results and your test code. ```python filename="git_version.py" import subprocess def get_git_info() -> dict: """Get current Git commit hash and tag.""" info = {} try: info["commit"] = subprocess.check_output( ["git", "rev-parse", "HEAD"], stderr=subprocess.DEVNULL, ).decode().strip() info["commit_short"] = subprocess.check_output( ["git", "rev-parse", "--short", "HEAD"], stderr=subprocess.DEVNULL, ).decode().strip() # Check for uncommitted changes status = subprocess.check_output( ["git", "status", "--porcelain"], stderr=subprocess.DEVNULL, ).decode().strip() info["dirty"] = len(status) > 0 # Get latest tag if available try: info["tag"] = subprocess.check_output( ["git", "describe", "--tags", "--abbrev=0"], stderr=subprocess.DEVNULL, ).decode().strip() except subprocess.CalledProcessError: info["tag"] = None except subprocess.CalledProcessError: info["commit"] = "unknown" info["commit_short"] = "unknown" info["dirty"] = True info["tag"] = None return info ``` ## Tag Test Runs with Code Version in TofuPilot Pass the Git info as metadata when you create a test run. Every run in TofuPilot will show which commit produced it. ```python filename="test_with_version.py" import openhtf as htf from openhtf.util import units from tofupilot.openhtf import TofuPilot from git_version import get_git_info git_info = get_git_info() # Warn if running with uncommitted changes in production if git_info["dirty"]: print("WARNING: Running tests with uncommitted changes") @htf.measures( htf.Measurement("voltage_3v3").in_range(3.2, 3.4).with_units(units.VOLT), ) def test_power_rail(test): test.measurements.voltage_3v3 = 3.29 def main(): test = htf.Test(test_power_rail) # Pass git info as run metadata with TofuPilot( test, script_version=git_info.get("tag") or git_info["commit_short"], ): test.execute(test_start=lambda: "DUT-001") if __name__ == "__main__": main() ``` Now when you look at a test run in TofuPilot, you can see which code version produced it. If a test starts failing after a deployment, compare the `script_version` of passing runs against failing ones to find the change. ## .gitignore Template for Test Projects Hardware test repos have specific files you don't want to track. Here's a practical starting point. ```python filename=".gitignore" # Python __pycache__/ *.py[cod] *.so *.egg-info/ dist/ build/ venv/ .venv/ # Environment and secrets .env .env.* !.env.example # IDE .vscode/ .idea/ *.swp *.swo # Test artifacts (logs, reports, captures) logs/ test_output/ *.log *.csv captures/ screenshots/ # Instrument calibration data (station-specific) cal_data/ fixture_cal/ # OS files .DS_Store Thumbs.db # PyInstaller (if you package tests) *.spec # Local config overrides config/local.yaml ``` A few notes on what's excluded: - **`.env` files** contain API keys. Never commit these. Include a `.env.example` with placeholder values so new team members know what to configure. - **`logs/` and `test_output/`** are generated per run. They belong in TofuPilot, not Git. - **`cal_data/`** is station-specific calibration data. It doesn't belong in the test script repo. - **`config/local.yaml`** is for per-developer overrides. The shared configs (`config/dev.yaml`, `config/production.yaml`) stay tracked. ### Track Rolled Throughput Yield URL: https://www.tofupilot.com/guides/how-to-track-rolled-throughput-yield-with-tofupilot Learn how to measure Rolled Throughput Yield (RTY) across multi-step manufacturing processes and track it in TofuPilot's dashboard. Final yield only tells you whether a unit passed at the end of the line. Rolled Throughput Yield (RTY) tells you the probability that a unit passes every step without rework or retesting. It's the metric that exposes hidden factory losses. ## Why RTY Matters More Than Final Yield Final yield counts a unit as "pass" even if it failed three times before someone reworked it. RTY doesn't. It multiplies the first-pass yield of every process step together: **RTY = FPY₁ x FPY₂ x ... x FPYₙ** Consider a five-step process where each step has 95% FPY. Final yield might show 99% because rework catches most failures. But RTY tells the real story: **RTY = 0.95 x 0.95 x 0.95 x 0.95 x 0.95 = 77.4%** That means nearly 1 in 4 units needed rework somewhere. Each rework event costs time, labor, and materials that final yield hides completely. ## RTY Interpretation | RTY Range | What It Means | |-----------|---------------| | > 95% | Process is well-controlled, minimal hidden rework | | 85-95% | Some steps need attention, review failure Pareto | | 70-85% | Significant hidden losses, prioritize worst FPY steps | | < 70% | Process is unstable, rework costs are likely substantial | The power of RTY is that it pinpoints which step drags down the whole line. If Step 3 has 88% FPY while others sit at 98%, you know exactly where to focus. ## Structuring Your Tests for RTY Tracking Each process step should be a separate test procedure in TofuPilot. This gives you per-step FPY that feeds directly into RTY calculations. Here's a multi-phase OpenHTF test that represents a PCB assembly process with distinct steps: ```python filename="test_smt_placement.py" # Step 1: SMT component placement verification import openhtf as htf from openhtf.util import units from tofupilot.openhtf import TofuPilot @htf.measures( htf.Measurement("placement_offset_x") .in_range(minimum=-0.05, maximum=0.05) .with_units(units.MILLIMETRE), htf.Measurement("placement_offset_y") .in_range(minimum=-0.05, maximum=0.05) .with_units(units.MILLIMETRE), htf.Measurement("component_rotation") .in_range(minimum=-2.0, maximum=2.0), ) def smt_placement_check(test): test.measurements.placement_offset_x = 0.02 test.measurements.placement_offset_y = -0.01 test.measurements.component_rotation = 0.5 def main(): test = htf.Test(smt_placement_check) with TofuPilot(test): test.execute(test_start=lambda: "PCB-2024-0042") if __name__ == "__main__": main() ``` ```python filename="test_reflow_solder.py" # Step 2: Reflow soldering thermal profile check import openhtf as htf from openhtf.util import units from tofupilot.openhtf import TofuPilot @htf.measures( htf.Measurement("peak_temperature") .in_range(minimum=235, maximum=250) .with_units(units.DEGREE_CELSIUS), htf.Measurement("time_above_liquidus") .in_range(minimum=60, maximum=120), htf.Measurement("cooling_rate") .in_range(maximum=3.0), ) def reflow_profile_check(test): test.measurements.peak_temperature = 242 test.measurements.time_above_liquidus = 85 test.measurements.cooling_rate = 2.1 def main(): test = htf.Test(reflow_profile_check) with TofuPilot(test): test.execute(test_start=lambda: "PCB-2024-0042") if __name__ == "__main__": main() ``` ```python filename="test_functional.py" # Step 3: Functional test after assembly import openhtf as htf from openhtf.util import units from tofupilot.openhtf import TofuPilot @htf.measures( htf.Measurement("supply_voltage") .in_range(minimum=4.9, maximum=5.1) .with_units(units.VOLT), htf.Measurement("quiescent_current") .in_range(maximum=15.0) .with_units(units.AMPERE), htf.Measurement("clock_frequency") .in_range(minimum=7.99, maximum=8.01) .with_units(units.HERTZ), ) def functional_check(test): test.measurements.supply_voltage = 5.02 test.measurements.quiescent_current = 0.0113 test.measurements.clock_frequency = 8003000 def main(): test = htf.Test(functional_check) with TofuPilot(test): test.execute(test_start=lambda: "PCB-2024-0042") if __name__ == "__main__": main() ``` Each test procedure runs independently and uploads results to TofuPilot with the same serial number. TofuPilot links all three procedures to the same unit, giving you a complete process history. ## Tracking RTY in TofuPilot Once your test procedures upload results, TofuPilot's dashboard gives you what you need to compute and track RTY: - **Per-procedure FPY trends** show first-pass yield for each process step over time. You can spot degradation in a single step before it affects final yield. - **Failure Pareto charts** break down which measurements fail most often within each step, so you can prioritize root cause analysis. - **Unit history** shows the complete journey of any serial number across all procedures, including retests. This is the ground truth for whether a unit passed first time at every step. You don't need to build RTY calculations in Python. Structure your tests as separate procedures, upload results with consistent serial numbers, and TofuPilot gives you the per-step visibility that makes RTY actionable. ## Key Practices for Accurate RTY 1. **One procedure per process step.** Don't combine SMT placement and reflow soldering into a single test. Separate procedures give you separate FPY numbers. 2. **Use consistent serial numbers.** Every procedure for the same unit must reference the same serial number. This is how TofuPilot links the full process chain. 3. **Don't filter out retests.** Upload every test execution, including failures and retests. RTY only works when you have the complete picture. 4. **Set measurement limits in code.** Limits defined in OpenHTF flow directly into TofuPilot's analytics. Don't rely on post-hoc analysis to determine pass/fail. ### Generate Test Reports for Auditors URL: https://www.tofupilot.com/guides/how-to-generate-test-reports-for-auditors-with-tofupilot Structure your OpenHTF tests so TofuPilot captures audit-ready records with serial numbers, measurements, limits, and metadata. Auditors want to see that every unit was tested, what was measured, what the limits were, and whether it passed. TofuPilot captures all of this automatically when you structure your OpenHTF tests correctly. ## What Auditors Need to See Regardless of the standard (ISO 9001, ISO 13485, IATF 16949, AS9100), audit requirements for test records share the same core elements. | Element | Why Auditors Ask for It | |---|---| | Serial number (DUT ID) | Proves traceability to a specific unit | | Timestamp | Proves when the test happened | | Station ID | Proves which equipment was used | | Operator | Proves who ran the test | | Measurements with limits | Proves acceptance criteria were applied | | Pass/fail verdict | Proves disposition was recorded | | Firmware or software version | Proves configuration at time of test | If your test records are missing any of these, you'll get a finding. TofuPilot stores all of them when you include them in your test run. ## Writing an Audit-Ready Test A good test record starts with a well-structured OpenHTF test. Include every measurement with explicit limits, and pass metadata through your test run. ```python filename="audit_ready_test.py" import openhtf as htf from openhtf.util import units from tofupilot.openhtf import TofuPilot @htf.measures( htf.Measurement("input_resistance") .in_range(minimum=950, maximum=1050) .with_units(units.OHM), htf.Measurement("leakage_current") .in_range(maximum=0.000005) .with_units(units.AMPERE), htf.Measurement("dielectric_strength_pass") .equals(True), ) def electrical_safety_test(test): test.measurements.input_resistance = 1002 test.measurements.leakage_current = 0.0000013 test.measurements.dielectric_strength_pass = True @htf.measures( htf.Measurement("output_power") .in_range(minimum=48.0, maximum=52.0) .with_units(units.WATT), htf.Measurement("efficiency_percent") .in_range(minimum=89.0), ) def performance_test(test): test.measurements.output_power = 50.1 test.measurements.efficiency_percent = 92.4 def main(): test = htf.Test( electrical_safety_test, performance_test, station_id="STATION-EOL-02", ) with TofuPilot(test): test.execute(test_start=lambda: "PSU-2026-00512") if __name__ == "__main__": main() ``` Every measurement gets stored with its value, limits, and verdict in TofuPilot. The serial number links the record to the unit's full history. ## What Makes a Good Test Record Three things separate a record that satisfies auditors from one that raises questions. **Consistent measurement names.** Use the same name for the same measurement everywhere. If `leakage_current` is called `leak_curr` on one station and `leakage_current` on another, you can't demonstrate consistent process monitoring. Pick one name and stick with it. **Explicit limits on every measurement.** A measured value without limits is just data. Auditors want to see that you defined acceptance criteria before production, not after. Always use `.in_range()` or `.equals()` on every measurement. **Meaningful metadata.** Include the operator, station, firmware version, and any other context your quality system defines. When an auditor asks "who tested unit X on what equipment with what software?", you need an answer. ## Viewing and Exporting Records Once your tests run, TofuPilot stores the complete record. You can access it in three ways. **Run detail page.** Search for any unit by serial number in TofuPilot's dashboard. The run detail page shows every measurement, its value, its limits, the verdict, timestamp, station, and all metadata. This is your primary view during an audit walkthrough. **CSV export.** Export run data from the dashboard for offline review or to attach to audit documentation. The export includes all measurements, limits, and metadata. **API access.** Pull run records programmatically through TofuPilot's REST API at `https://tofupilot.app` for integration with your QMS or document control system. This is useful if your quality team needs to generate batch records or certificates of conformance. ## Metadata Checklist Before going into production, verify that your tests capture everything your quality system requires. Here's a starting checklist. - [ ] DUT serial number (unique per unit) - [ ] Test station ID (matches calibration records) - [ ] Operator ID (matches training records) - [ ] Firmware/software version (matches release records) - [ ] All measurements have limits defined - [ ] Measurement names are consistent across stations - [ ] Test procedure name or ID is included - [ ] Work order or batch number (if applicable) If you're missing any of these, add them to your measurement definitions before production starts. It's much harder to backfill traceability data after the fact. ### Export Test Data to Power BI URL: https://www.tofupilot.com/guides/how-to-export-test-data-to-power-bi-with-tofupilot Learn how to export test data from TofuPilot to Power BI using CSV exports and the REST API for custom management reports and cross-system analysis. TofuPilot's built-in dashboards cover most manufacturing analytics needs. But sometimes you need to combine test data with ERP, MES, or supply chain data in a single report for management. That's when Power BI comes in. ## When You Need External BI TofuPilot handles FPY trends, Cpk, control charts, failure Pareto, and throughput natively. You don't need Power BI for any of those. External BI tools make sense when you need to correlate test results with data from other systems. Examples: mapping yield by supplier lot (ERP data), comparing test costs to warranty claims (finance data), or building executive summaries that pull from five different sources. ## Option 1: CSV Export The fastest way to get data into Power BI. TofuPilot lets you export test runs and measurements as CSV files directly from the dashboard. In Power BI Desktop, use **Get Data > Text/CSV** to import the file. This works well for one-time reports or periodic snapshots. For recurring reports, save the CSV to a shared folder and configure Power BI to refresh from that location on a schedule. ## Option 2: REST API For automated, always-fresh data, connect Power BI directly to TofuPilot's REST API. This pulls the latest test data every time the report refreshes. ### Getting Your API Key Generate an API key at [https://tofupilot.app](https://tofupilot.app) under your workspace settings. Store it securely. You'll pass it as a header in every request. ### API Endpoints TofuPilot's API provides endpoints for the data you'll need in Power BI: - `GET /api/v1/runs` returns test runs with status, timestamps, DUT ID, and procedure info - `GET /api/v1/runs/{id}` returns a single run with full measurement data The API returns JSON. Paginate through results using the `offset` and `limit` query parameters. ```python filename="fetch_runs_example.py" import requests API_KEY = "your-api-key" BASE_URL = "https://tofupilot.app/api/v1" response = requests.get( f"{BASE_URL}/runs", headers={"Authorization": f"Bearer {API_KEY}"}, params={"limit": 100, "offset": 0}, ) runs = response.json() ``` ### Connecting Power BI to the API In Power BI Desktop, use **Get Data > Web** and configure the request: 1. Select **Advanced** in the Web dialog 2. Set the URL to `https://tofupilot.app/api/v1/runs?limit=100` 3. Add the header `Authorization` with value `Bearer your-api-key` 4. Click **OK** to load the JSON response Power BI's Power Query editor will show the raw JSON. Use **To Table > Expand Columns** to flatten the nested structure into rows and columns. ### Handling Pagination TofuPilot's API paginates results. To load all runs, create a parameterized query in Power Query that iterates through pages: ```text filename="power_query_pagination.m" let BaseUrl = "https://tofupilot.app/api/v1/runs", PageSize = 100, GetPage = (offset) => let Response = Web.Contents( BaseUrl, [ Query = [limit = Text.From(PageSize), offset = Text.From(offset)], Headers = [Authorization = "Bearer your-api-key"] ] ), Json = Json.Document(Response) in Json, AllPages = List.Generate( () => [Page = GetPage(0), Offset = 0], each List.Count([Page]) > 0, each [Page = GetPage([Offset] + PageSize), Offset = [Offset] + PageSize], each [Page] ), Combined = List.Combine(AllPages), AsTable = Table.FromList(Combined, Splitter.SplitByNothing(), null, null, ExtraValues.Error) in AsTable ``` This query fetches pages until it gets an empty result, then combines everything into one table. ## Building Useful Reports Once the data is in Power BI, keep the reports focused on what TofuPilot doesn't already show. **Cross-system correlation.** Join test runs with supplier lot data from your ERP. Build a matrix of yield by supplier, by component, by date range. This answers "which supplier's parts cause the most failures?" without manual spreadsheet work. **Executive summaries.** Combine yield metrics with production volume and cost data. Show units shipped, test cost per unit, and warranty return rate on a single page. **Multi-site comparison.** If you run TofuPilot across multiple factories, Power BI can pull from all instances and normalize the data into a single cross-site view. ## Keeping Data Fresh Set up scheduled refresh in Power BI Service (cloud) to pull new data daily or hourly. For the API connection, store credentials in Power BI's data source settings so refreshes run unattended. For CSV-based reports, automate the export with a script that runs on a schedule and drops the file where Power BI expects it. ### Test Traceability for Medical Devices URL: https://www.tofupilot.com/guides/test-traceability-for-medical-devices-fda-with-tofupilot Map FDA 21 CFR Part 820 and ISO 13485 requirements to TofuPilot features for device history records, test documentation, and audit trails. FDA 21 CFR Part 820 requires medical device manufacturers to maintain a Device History Record (DHR) for every unit produced. TofuPilot captures test results, serial numbers, measurements, and pass/fail determinations in an immutable record that maps directly to FDA and ISO 13485 requirements. ## Regulatory Requirements Overview Medical device test traceability sits at the intersection of two frameworks. Both require documented evidence that each device was manufactured and tested according to approved procedures. | Regulation | Clause | Requirement | TofuPilot Feature | |---|---|---|---| | 21 CFR 820.184 | DHR | Records demonstrating device manufactured per DMR | Test records per serial number with procedure name | | 21 CFR 820.80 | Receiving, in-process, finished device acceptance | Acceptance activities with acceptance/rejection documented | Pass/fail results with measurement limits | | 21 CFR 820.90 | Nonconforming product | Documented investigation of nonconformances | Failed runs with measurement-level detail | | 21 CFR 820.180 | General requirements (records) | Records maintained for device lifetime or 2 years | Immutable cloud storage with retention | | 21 CFR 820.250 | Statistical techniques | Valid statistical techniques for process capability | Cpk, control charts, FPY in dashboard | | ISO 13485 7.5.3 | Traceability | Records per unit or batch through production | Serial tracking, unit history, sub-assembly links | | ISO 13485 8.2.4 | Monitoring and measurement | Evidence of conformity to acceptance criteria | Measurements with units, limits, and pass/fail | ## Structuring Tests for DHR Compliance The DHR must include test results that prove each device was manufactured according to its Device Master Record (DMR). Structure your OpenHTF tests so each procedure maps to a documented test protocol. ```python filename="test_medical_device.py" # Medical device production test aligned with DHR requirements import openhtf as htf from openhtf.util import units from tofupilot.openhtf import TofuPilot @htf.measures( htf.Measurement("impedance_channel_1") .in_range(minimum=980, maximum=1020) .with_units(units.OHM) .doc("Electrode impedance per IEC 60601-1, channel 1"), htf.Measurement("impedance_channel_2") .in_range(minimum=980, maximum=1020) .with_units(units.OHM) .doc("Electrode impedance per IEC 60601-1, channel 2"), htf.Measurement("leakage_current") .in_range(maximum=0.00001) .with_units(units.AMPERE) .doc("Patient leakage current, Type BF applied part per IEC 60601-1"), ) def electrical_safety(test): test.measurements.impedance_channel_1 = 1002.3 test.measurements.impedance_channel_2 = 997.1 test.measurements.leakage_current = 0.000003 @htf.measures( htf.Measurement("signal_accuracy_pct") .in_range(minimum=99.0, maximum=101.0) .doc("Measurement accuracy against NIST-traceable reference"), htf.Measurement("noise_floor") .in_range(maximum=0.000005) .with_units(units.VOLT) .doc("Input-referred noise, 0.05-150 Hz bandwidth"), htf.Measurement("common_mode_rejection") .in_range(minimum=100) .doc("CMRR at 50/60 Hz, in dB"), ) def signal_performance(test): test.measurements.signal_accuracy_pct = 99.8 test.measurements.noise_floor = 0.0000021 test.measurements.common_mode_rejection = 112.4 @htf.measures( htf.Measurement("firmware_version") .doc("Installed firmware version, must match DMR revision"), htf.Measurement("self_test_result") .in_range(minimum=1, maximum=1) .doc("Device self-test pass (1) or fail (0)"), htf.Measurement("battery_capacity_pct") .in_range(minimum=95) .doc("Battery capacity relative to rated"), ) def system_validation(test): test.measurements.firmware_version = "3.2.1-release" test.measurements.self_test_result = 1 test.measurements.battery_capacity_pct = 98.7 def main(): test = htf.Test( electrical_safety, signal_performance, system_validation, ) with TofuPilot(test): test.execute(test_start=lambda: "MD-2026-04-00192") if __name__ == "__main__": main() ``` Each measurement references the applicable standard and includes acceptance limits. TofuPilot stores the procedure name, the serial number (UDI-linked), and every measurement with its result. ## Acceptance Activities (21 CFR 820.80) The FDA requires documented acceptance activities at receiving, in-process, and finished device stages. Run separate OpenHTF procedures for each stage, all using the same serial number. ```python filename="test_acceptance_stages.py" # Three-stage acceptance testing for FDA 820.80 compliance import openhtf as htf from openhtf.util import units from tofupilot.openhtf import TofuPilot @htf.measures( htf.Measurement("incoming_visual_pass") .in_range(minimum=1, maximum=1), htf.Measurement("incoming_dimension_check") .in_range(minimum=24.9, maximum=25.1) .with_units(units.MILLIMETRE), ) def receiving_inspection(test): test.measurements.incoming_visual_pass = 1 test.measurements.incoming_dimension_check = 25.02 @htf.measures( htf.Measurement("solder_paste_height") .in_range(minimum=0.10, maximum=0.15) .with_units(units.MILLIMETRE), htf.Measurement("component_presence") .in_range(minimum=1, maximum=1), ) def in_process_inspection(test): test.measurements.solder_paste_height = 0.12 test.measurements.component_presence = 1 @htf.measures( htf.Measurement("final_functional_test") .in_range(minimum=1, maximum=1), htf.Measurement("label_verification") .in_range(minimum=1, maximum=1), ) def finished_device_acceptance(test): test.measurements.final_functional_test = 1 test.measurements.label_verification = 1 def run_stage(phase, serial): test = htf.Test(phase) with TofuPilot(test): test.execute(test_start=lambda: serial) def main(): serial = "MD-2026-04-00192" run_stage(receiving_inspection, serial) run_stage(in_process_inspection, serial) run_stage(finished_device_acceptance, serial) if __name__ == "__main__": main() ``` TofuPilot's unit page for this serial shows all three acceptance stages in order. An FDA inspector can see the complete manufacturing and test history from a single page. ## Nonconforming Product (21 CFR 820.90) When a device fails testing, the FDA requires documented investigation. TofuPilot records the exact measurement that triggered the failure, its value, and the limit it exceeded. Failed test runs are immediately visible in TofuPilot's dashboard. The unit's history page shows every attempt, so you can track the investigation, rework, and retest cycle. This satisfies the 820.90 requirement to document the nonconformance, investigation, and disposition. ## Component Traceability with Sub-Units ISO 13485 clause 7.5.3 requires traceability of components used in each device. Use TofuPilot's `sub_units` feature to link component serial numbers to the finished device. ```python filename="test_component_traceability.py" # Record component serials for ISO 13485 traceability import openhtf as htf from tofupilot.openhtf import TofuPilot @htf.measures( htf.Measurement("assembly_complete") .in_range(minimum=1, maximum=1), ) def final_assembly(test): test.measurements.assembly_complete = 1 def main(): test = htf.Test(final_assembly) with TofuPilot( test, sub_units=[ {"serial_number": "SENSOR-2026-08812"}, {"serial_number": "PCBA-2026-04410"}, {"serial_number": "BATT-2026-11023"}, ], ): test.execute(test_start=lambda: "MD-2026-04-00192") if __name__ == "__main__": main() ``` If a component supplier issues a recall, you can trace which finished devices contain affected components through TofuPilot's unit hierarchy. ## Audit Trail and Exports TofuPilot provides the documentation that FDA and ISO 13485 auditors expect: - **Device History Record.** Every test run for a serial number, with procedure, station, operator, measurements, and result. This forms the test portion of the DHR. - **Process capability.** Cpk values and control charts for any measurement, demonstrating statistical process control per 21 CFR 820.250. - **Failure analysis.** Failure Pareto and measurement histograms to support CAPA investigations. - **Record retention.** Immutable storage with no ability to alter or delete historical records through the standard interface. Export test data from the dashboard for inclusion in your formal DHR documentation package. Records include timestamps, station identifiers, and all measurement values with their acceptance criteria. ### Replace Excel Test Tracking URL: https://www.tofupilot.com/guides/how-to-replace-excel-test-tracking-with-tofupilot Move from spreadsheet-based test tracking to structured, automated test data collection with OpenHTF and TofuPilot. Most hardware teams start tracking test results in Excel or Google Sheets. It works for a while, then it doesn't. You lose real-time visibility, can't handle concurrent access, and manual entry introduces errors that compound over time. TofuPilot replaces that workflow with structured, automated test data collection. You keep writing tests in Python, and TofuPilot handles storage, analytics, and traceability. ## The Spreadsheet Pattern A typical Excel test log looks something like this: | Serial Number | Date | Operator | Result | Voltage (V) | Current (A) | Firmware | Notes | |---|---|---|---|---|---|---|---| | SN-001 | 2025-01-15 | Alice | PASS | 3.31 | 0.52 | v2.1 | | | SN-002 | 2025-01-15 | Bob | FAIL | 3.58 | 0.89 | v2.1 | Over current limit | | SN-003 | 2025-01-16 | Alice | PASS | 3.29 | 0.48 | v2.1 | | This format has real problems at scale: - **No concurrent access.** Two operators can't log results simultaneously without risking overwrites or merge conflicts. - **No validation.** Nothing stops someone from entering "PSAS" instead of "PASS" or putting voltage in the current column. - **No analytics.** Calculating FPY, Cpk, or failure trends means writing fragile formulas or pivot tables that break when the sheet structure changes. - **No history.** When someone edits a cell, the original value is gone. You have no audit trail. ## The OpenHTF + TofuPilot Equivalent Here's the same test expressed as an OpenHTF test with TofuPilot integration. Every run is automatically structured, validated, and stored. ```python filename="power_board_test.py" import openhtf as htf from openhtf.util import units from tofupilot.openhtf import TofuPilot @htf.measures( htf.Measurement("voltage").with_units(units.VOLT).in_range(3.1, 3.5), htf.Measurement("current").with_units(units.AMPERE).in_range(maximum=0.7), ) def power_supply_check(test): voltage = 3.31 # Read from your instrument current = 0.52 test.measurements.voltage = voltage test.measurements.current = current def main(): test = htf.Test(power_supply_check) with TofuPilot(test): test.execute(test_start=lambda: "SN-001") if __name__ == "__main__": main() ``` Each run automatically captures the serial number, pass/fail outcome, every measurement with its limits, timestamps, and the test station identity. No manual entry. No copy-paste errors. ## What Changes When You Switch | Capability | Excel / Google Sheets | TofuPilot | |---|---|---| | Data entry | Manual, error-prone | Automatic from test code | | Concurrent access | File locks, merge conflicts | Multi-user, multi-station by default | | Measurement validation | None | Limits enforced at test time | | FPY and yield trends | Manual formulas | Built-in dashboard, real-time | | Cpk and SPC | Requires custom macros | Automatic control charts | | Failure analysis | Manual filtering | Failure Pareto, drill-down by station | | Audit trail | No history | Full revision history per run | | Search and filter | Ctrl+F | Filter by serial, station, date, outcome, batch | | API access | None | REST API for integrations | ## Keeping Your Existing Data If you have historical test data in spreadsheets that you want to preserve, you can import it through TofuPilot's REST API. Structure each row as a test run with measurements and POST it to the API. ```python filename="import_from_csv.py" import csv from tofupilot import TofuPilotClient client = TofuPilotClient() with open("test_results.csv") as f: reader = csv.DictReader(f) for row in reader: client.create_run( procedure_id="power-board-test", unit_under_test={"serial_number": row["Serial Number"]}, run_passed=row["Result"] == "PASS", steps=[ { "name": "power_supply_check", "step_passed": row["Result"] == "PASS", "measurements": [ { "name": "voltage", "measured_value": float(row["Voltage (V)"]), "unit": "V", "lower_limit": 3.1, "upper_limit": 3.5, }, { "name": "current", "measured_value": float(row["Current (A)"]), "unit": "A", "upper_limit": 0.7, }, ], } ], ) ``` Run this once to backfill your history, then switch all new tests to the OpenHTF workflow. ## What You Get in the Dashboard Once your tests report to TofuPilot, open the dashboard at [tofupilot.app](https://tofupilot.app). You'll find: - **FPY trends** across stations and time periods, calculated automatically. - **Measurement histograms** with Cpk and control charts for every measurement you define. - **Failure Pareto** showing which measurements fail most often and on which stations. - **Station throughput** so you can see which lines are running and which are idle. - **Full traceability** per serial number, with every test run, measurement, and revision linked. These are the analytics you'd otherwise build with pivot tables, VBA macros, or custom scripts. They update in real time as new runs come in. ## Running Both Systems in Parallel You don't have to switch overnight. A practical migration path: 1. **Pick one test station** and add the TofuPilot integration to its OpenHTF tests. 2. **Run both systems** for a week. Keep logging to the spreadsheet while TofuPilot collects the same data automatically. 3. **Compare results** to build confidence that nothing is lost. 4. **Roll out** to remaining stations once you're satisfied. The spreadsheet stays as a backup until you're ready to retire it. No data is at risk during the transition. ### What Is First Pass Yield and How to Track It URL: https://www.tofupilot.com/guides/what-is-first-pass-yield-and-how-to-track-it-with-tofupilot Learn how to calculate first pass yield (FPY), why it matters for manufacturing test, and how to track it automatically with TofuPilot. First pass yield is the percentage of units that pass all tests on the first attempt, without rework or retesting. It's the single most important metric for manufacturing test efficiency. A low FPY means wasted time, wasted parts, and hidden quality problems. A high FPY means your process works. ## How to Calculate First Pass Yield The formula is simple: | Variable | Meaning | |----------|---------| | FPY | First pass yield (0 to 1, or 0% to 100%) | | Units passed | Units that passed all tests on the first run | | Total units tested | All units that entered the test process | **FPY = Units passed on first attempt / Total units tested** If you tested 1,000 PCBAs and 950 passed on the first attempt, your FPY is 95%. For a multi-step process with N stations, the **rolled throughput yield (RTY)** multiplies each station's FPY: | Station | FPY | |---------|-----| | ICT | 98% | | Functional Test | 95% | | Final Assembly | 99% | | **RTY** | **98% x 95% x 99% = 92.2%** | RTY shows the real probability that a unit passes the entire line without rework. Even when individual stations look healthy, the rolled yield often tells a different story. ## Why FPY Matters More Than You Think ### The cost of retesting Every failed unit costs more than the retest itself. Here's what actually happens when a unit fails: 1. **Operator time** to remove, label, and log the failure 2. **Diagnostic time** to identify the root cause 3. **Rework cost** (soldering, component replacement, firmware reflash) 4. **Retest cost** (the unit goes through the station again) 5. **Throughput loss** (the station was occupied by a unit that should've passed) A test station running at 95% FPY wastes 5% of its capacity on retests. At scale, that's a full station worth of throughput lost. Most teams underestimate this because they don't track the total cost per failure. ### FPY benchmarks by industry | Industry | Typical FPY | World-class FPY | |----------|------------|----------------| | Consumer electronics | 95-98% | >99% | | Automotive electronics | 97-99% | >99.5% | | Medical devices | 90-95% | >98% | | Aerospace / defense | 85-95% | >97% | | IoT / sensors | 93-97% | >99% | These numbers vary by product complexity, but they give you a reference point. If you're below the typical range, there's likely a process issue worth investigating. ## Common Causes of Low FPY Before you can improve FPY, you need to know what's failing and why. The most common causes in electronics manufacturing: ### Test-related (false failures) - **Limits set too tight.** Initial limits from the datasheet don't account for real production variation. A limit that rejects 3% of units might be catching normal variation, not defects. - **Fixture contact issues.** Worn probes, misaligned pins, or contaminated contacts cause intermittent failures that pass on retest. - **Environmental sensitivity.** Temperature drift in the test station causes measurements to shift during a production run. ### Process-related (real failures) - **Solder defects.** Cold joints, bridges, insufficient solder. The #1 source of functional test failures in SMT production. - **Component placement errors.** Wrong orientation, tombstoning, shifted components. - **Supplier variation.** A new component lot with slightly different characteristics triggers marginal failures. The first step is separating false failures from real ones. If a unit fails then passes on retest with no rework, it was probably a false failure. Tracking retest pass rates helps you quantify this. ## How to Track FPY with TofuPilot TofuPilot calculates FPY automatically from your test data. Every test run you upload includes a pass/fail result, and TofuPilot aggregates these into FPY metrics by procedure, time period, and station. ### Basic test script Here's a minimal OpenHTF test that logs results to TofuPilot. FPY tracking starts automatically once you have runs flowing in. ```python filename="fpy_example/main.py" import openhtf as htf from openhtf.util import units from tofupilot.openhtf import TofuPilot @htf.measures( htf.Measurement("supply_voltage_3v3") .in_range(3.2, 3.4) .with_units(units.VOLT), htf.Measurement("supply_current_idle") .in_range(0.01, 0.15) .with_units(units.AMPERE), ) def test_power_rails(test): """Verify 3.3V rail voltage and idle current draw.""" # Replace with actual instrument readings test.measurements.supply_voltage_3v3 = 3.31 test.measurements.supply_current_idle = 0.042 @htf.measures( htf.Measurement("firmware_version") .equals("2.1.0"), htf.Measurement("self_test_result") .equals("PASS"), ) def test_firmware(test): """Check firmware version and run DUT self-test.""" # Replace with actual DUT communication test.measurements.firmware_version = "2.1.0" test.measurements.self_test_result = "PASS" def main(): test = htf.Test( test_power_rails, test_firmware, procedure_id="FCT-001", part_number="PCBA-2024-A", ) with TofuPilot(test): test.execute(test_start=lambda: input("Scan serial number: ")) if __name__ == "__main__": main() ``` Each run is logged with its serial number, measurements, limits, and pass/fail status. TofuPilot uses this data to compute FPY in real time. ### What you get automatically Once test runs flow into TofuPilot, the procedure analytics page shows: - **FPY over time** with daily, weekly, and monthly views - **FPY by station** to compare performance across test stations - **Failure Pareto** showing which measurements cause the most failures - **Control charts** for each measurement with 3-sigma limits - **Cpk values** showing process capability relative to your test limits No extra code needed. These analytics are computed from the measurements and limits you already define in your test script. ## How to Improve FPY ### Step 1: Find the top failure modes Open the procedure analytics in TofuPilot and sort failures by frequency. The Pareto chart shows which measurements cause the most failures. Focus on the top 3. In most production lines, 2-3 failure modes account for 80% of all failures. ### Step 2: Separate false failures from real ones For each top failure mode, check the retest behavior: | Retest Behavior | Likely Cause | Action | |----------------|-------------|--------| | Passes on retest, no rework | False failure (contact, noise, timing) | Fix the test, not the product | | Passes on retest after rework | Real defect caught correctly | Improve upstream process | | Fails again on retest | Consistent defect | Likely design or component issue | False failures are the easiest wins. Tightening fixture maintenance schedules or adding measurement averaging can recover 1-3% FPY overnight. ### Step 3: Refine test limits with production data Initial limits from the datasheet are a starting point. After running 500+ units, use TofuPilot's control charts to see the actual distribution of each measurement. Set limits at mean +/- 3 sigma from your production data, constrained by the datasheet spec. This catches two problems: - **Limits too tight** reject good units (false failures, lower FPY) - **Limits too loose** miss defective units (escapes, field failures) TofuPilot's control charts show both the current limits and the 3-sigma values, so you can see where they diverge. ### Step 4: Monitor trends FPY isn't static. It changes when you switch component suppliers, adjust solder paste profiles, or deploy new firmware. Set up a weekly review of FPY trends in TofuPilot. A sudden drop usually points to a specific event you can trace. ## FPY vs Other Quality Metrics | Metric | What It Measures | When to Use | |--------|-----------------|-------------| | **FPY** | % passing on first attempt | Overall test efficiency | | **RTY** | Cumulative yield across all stations | End-to-end process health | | **Cpk** | Process capability vs spec limits | Individual measurement stability | | **DPMO** | Defects per million opportunities | Six Sigma programs, supplier comparison | | **OEE** | Overall equipment effectiveness | Station utilization analysis | FPY is the simplest to track and the most actionable. Start here. Add Cpk when you need to analyze specific measurements. Use RTY when you have multiple test stations in sequence. ### How to Migrate from WATS to TofuPilot URL: https://www.tofupilot.com/guides/how-to-migrate-from-wats-to-tofupilot Step-by-step migration from WATS to TofuPilot, covering concept mapping, data migration, and parallel running. If you're evaluating a WATS alternative, this guide walks through the migration path. Common triggers include cost (WATS Analytics runs at EUR 297/mo per module), a preference for Python-first workflows, or wanting to build on an open-source test framework instead of proprietary data converters. TofuPilot pairs with OpenHTF (Google's open-source test framework) to give you structured test data, real-time analytics, and a REST API without vendor lock-in on the test execution side. ## Concept Mapping WATS and TofuPilot use different terminology for similar concepts. Here's how they map: | WATS Concept | TofuPilot Equivalent | Notes | |---|---|---| | UUT Report | Test Run | One execution of a test sequence on a unit | | Test Step | Phase | A phase in OpenHTF, containing measurements | | Numeric Limit Test | Measurement with limits | Defined via `htf.Measurement().in_range()` | | String Value Test | Measurement (string) | Any measured value, not just numeric | | WATS Client / Data Converter | TofuPilot Python SDK | `from tofupilot.openhtf import TofuPilot` | | WATS Dashboard | TofuPilot Dashboard | FPY, Cpk, Pareto, control charts at [tofupilot.app](https://tofupilot.app) | | Test Sequence File | OpenHTF Test | Python script defining phases and measurements | | Station Registration | Automatic | TofuPilot detects stations from test metadata | ## Replacing the WATS Data Converter With WATS, you typically use a data converter (TestStand, LabVIEW, or custom) to serialize test results and push them to the WATS API. With TofuPilot, you write your test logic in OpenHTF and the SDK handles reporting directly. Here's a typical test that would replace a WATS data converter workflow: ```python filename="functional_test.py" import openhtf as htf from openhtf.util import units from tofupilot.openhtf import TofuPilot @htf.measures( htf.Measurement("supply_voltage") .with_units(units.VOLT) .in_range(4.75, 5.25), htf.Measurement("clock_frequency") .with_units(units.HERTZ) .in_range(minimum=7.99e6), htf.Measurement("firmware_version"), ) def power_and_clock_check(test): test.measurements.supply_voltage = 5.02 test.measurements.clock_frequency = 8.00e6 test.measurements.firmware_version = "v3.2.1" @htf.measures( htf.Measurement("signal_amplitude") .with_units(units.VOLT) .in_range(0.9, 1.1), htf.Measurement("signal_thd") .in_range(maximum=0.05), ) def signal_integrity_check(test): test.measurements.signal_amplitude = 1.01 test.measurements.signal_thd = 0.03 def main(): test = htf.Test(power_and_clock_check, signal_integrity_check) with TofuPilot(test): test.execute(test_start=lambda: "SN-10042") if __name__ == "__main__": main() ``` No data converter needed. The TofuPilot SDK serializes the OpenHTF test record and uploads it automatically. Each phase maps to what WATS calls a test step, and each measurement carries its value, units, and limits. ## Migrating Historical Data You likely have test history in WATS that you want to preserve. TofuPilot's REST API accepts historical runs with explicit timestamps, so you can backfill without losing chronological context. First, export your data from WATS (CSV export or API). Then import it: ```python filename="import_wats_history.py" import csv from datetime import datetime from tofupilot import TofuPilotClient client = TofuPilotClient() with open("wats_export.csv") as f: reader = csv.DictReader(f) for row in reader: client.create_run( procedure_id="functional-test", unit_under_test={ "serial_number": row["SerialNumber"], "part_number": row["PartNumber"], }, run_passed=row["Status"] == "Passed", started_at=datetime.fromisoformat(row["StartTime"]), duration=float(row["Duration"]), steps=[ { "name": row["StepName"], "step_passed": row["StepStatus"] == "Passed", "measurements": [ { "name": row["MeasurementName"], "measured_value": float(row["Value"]), "unit": row["Unit"], "lower_limit": float(row["LowLimit"]) if row["LowLimit"] else None, "upper_limit": float(row["HighLimit"]) if row["HighLimit"] else None, } ], } ], ) ``` For complex WATS exports with nested test steps, flatten them into TofuPilot's phase/measurement structure. Each WATS numeric limit test becomes a measurement with `lower_limit` and `upper_limit`. ## Parallel Running Strategy Don't cut over in one shot. Run both systems side by side to validate the migration: 1. **Week 1.** Pick one test station. Add the TofuPilot integration to its tests while keeping the WATS data converter active. Both systems receive the same data. 2. **Week 2.** Compare results in both dashboards. Verify that measurement values, pass/fail outcomes, and station attribution match. 3. **Week 3.** If everything aligns, disable the WATS data converter on that station. Roll out to the next station. 4. **Repeat** until all stations report to TofuPilot only. Keep your WATS subscription active during this period. You can export a final data snapshot before canceling. ## What You Get After Migration Once your tests report to TofuPilot, you'll find the analytics at [tofupilot.app](https://tofupilot.app): - **FPY trends** by station, product, and time range. - **Cpk and control charts** for every measurement, computed automatically. - **Failure Pareto** to identify your top failure modes. - **Measurement histograms** showing distribution against limits. - **Full traceability** per serial number across all test runs. - **REST API** for integrating with your MES, ERP, or custom tooling. These replace the WATS Analytics module. The difference is that your test execution layer is now open-source (OpenHTF), and you're not locked into a proprietary test framework to get analytics. ### The True Cost of Poor Quality in Electronics URL: https://www.tofupilot.com/guides/the-true-cost-of-poor-quality-in-electronics-with-tofupilot Understand COPQ categories, the 1-10-100 rule for defect costs, and how proper test coverage with TofuPilot catches issues before they become expensive. Every defective unit has a price tag, and it grows the further that defect travels. The Cost of Poor Quality (COPQ) captures everything you spend because something wasn't built right the first time. For most electronics manufacturers, COPQ runs between 15% and 25% of revenue. Most of it is invisible. ## What COPQ Includes COPQ splits into two categories: internal failures (caught before shipping) and external failures (caught by the customer). ### Internal Failure Costs These happen inside your factory. They're painful but controllable. | Cost Type | Example | Typical Impact | |-----------|---------|---------------| | Scrap | PCB with tombstoned components sent to recycling | Full material + labor cost lost | | Rework | Re-soldering a BGA after X-ray finds voids | Labor + equipment time + retest | | Retesting | Unit fails functional test, gets retested after fix | Station time blocked, throughput drops | | Downgrading | Unit doesn't meet Grade A spec, sold as Grade B | Revenue loss per unit | | Failure analysis | Engineering time to diagnose root cause | Hours of skilled labor diverted | ### External Failure Costs These happen after the product ships. They're where the real damage lives. | Cost Type | Example | Typical Impact | |-----------|---------|---------------| | Warranty claims | Customer returns a dead power supply | Replacement unit + shipping + processing | | Field service | Technician dispatched to replace a failed module | Travel + labor + downtime penalty | | Recalls | Batch of units with defective firmware update | Logistics + PR + regulatory reporting | | Customer churn | Customer switches to competitor after repeated issues | Lifetime revenue lost | | Brand damage | Negative reviews and lost referrals | Hard to quantify, slow to recover | ## The 1-10-100 Rule This rule, originally from quality management literature, puts a ratio on when you catch a defect: - **$1 for prevention.** Design the test, validate the process, set the limits. A well-written OpenHTF test with proper measurement limits costs almost nothing per unit to run. - **$10 for detection.** Catch the defect on the line. You've already spent material and labor, but you can rework or scrap before shipping. This is where test stations earn their keep. - **$100 for failure.** The defect reaches the customer. Now you're paying for returns, field service, warranty processing, and the trust you can't invoice for. The ratios vary by industry. In medical devices, external failure costs can be 1000x prevention costs when you factor in regulatory consequences. In consumer electronics, it's closer to the 1-10-100 model. The principle is always the same: catch it earlier, pay less. ## Where Test Coverage Fits Test coverage is your primary tool for converting $100 problems into $10 problems, and $10 problems into $1 investments. ### Coverage Gaps Cost Money Consider a Bluetooth speaker manufacturer running only a final functional test. They check audio output and pairing. What they don't catch: - Cold solder joints that pass at room temperature but fail after thermal cycling - Battery cells with slightly low capacity that die early in the field - Antenna impedance mismatch that causes range issues in certain orientations Each gap is a field failure waiting to happen. Adding targeted tests at earlier stages closes these gaps. ### Structuring Tests to Catch Defects Early A well-structured test strategy pushes detection upstream. Here's what that looks like for an IoT sensor module: ```python filename="test_incoming_inspection.py" # Incoming inspection: catch component issues before assembly import openhtf as htf from tofupilot.openhtf import TofuPilot @htf.measures( htf.Measurement("crystal_frequency") .in_range(minimum=31.990, maximum=32.010), htf.Measurement("sensor_ic_id_register") .equals(0xB5), htf.Measurement("flash_chip_capacity_mb") .equals(16), ) def incoming_component_check(test): test.measurements.crystal_frequency = 32.001 test.measurements.sensor_ic_id_register = 0xB5 test.measurements.flash_chip_capacity_mb = 16 def main(): test = htf.Test(incoming_component_check) with TofuPilot(test): test.execute(test_start=lambda: "IOT-2024-4401") if __name__ == "__main__": main() ``` ```python filename="test_post_assembly.py" # Post-assembly test: validate solder quality and basic function import openhtf as htf from openhtf.util import units from tofupilot.openhtf import TofuPilot @htf.measures( htf.Measurement("supply_current_sleep") .in_range(maximum=0.000050) .with_units(units.AMPERE), htf.Measurement("supply_current_active") .in_range(minimum=0.015, maximum=0.035) .with_units(units.AMPERE), htf.Measurement("i2c_sensor_ack") .equals(True), htf.Measurement("spi_flash_read_write_ok") .equals(True), ) def post_assembly_validation(test): test.measurements.supply_current_sleep = 0.000028 test.measurements.supply_current_active = 0.0243 test.measurements.i2c_sensor_ack = True test.measurements.spi_flash_read_write_ok = True def main(): test = htf.Test(post_assembly_validation) with TofuPilot(test): test.execute(test_start=lambda: "IOT-2024-4401") if __name__ == "__main__": main() ``` ```python filename="test_final_calibration.py" # Final calibration and functional test import openhtf as htf from openhtf.util import units from tofupilot.openhtf import TofuPilot @htf.measures( htf.Measurement("temperature_accuracy") .in_range(minimum=-0.5, maximum=0.5) .with_units(units.DEGREE_CELSIUS), htf.Measurement("humidity_accuracy_pct") .in_range(minimum=-3.0, maximum=3.0), htf.Measurement("ble_rssi_at_1m") .in_range(minimum=-55, maximum=-35), htf.Measurement("battery_voltage") .in_range(minimum=3.0, maximum=4.2) .with_units(units.VOLT), htf.Measurement("ota_update_success") .equals(True), ) def final_calibration_test(test): test.measurements.temperature_accuracy = 0.12 test.measurements.humidity_accuracy_pct = -1.4 test.measurements.ble_rssi_at_1m = -42 test.measurements.battery_voltage = 3.85 test.measurements.ota_update_success = True def main(): test = htf.Test(final_calibration_test) with TofuPilot(test): test.execute(test_start=lambda: "IOT-2024-4401") if __name__ == "__main__": main() ``` Three test stages, each catching a different class of defect. Incoming inspection catches bad components before you solder them (cheapest fix). Post-assembly catches process defects before calibration effort is wasted. Final test catches system-level issues before shipping. ## Using TofuPilot to Track Quality Costs You can't reduce COPQ without measuring it. TofuPilot gives you the data foundation: - **FPY per procedure** shows your first-pass yield at each test stage. A drop in FPY at post-assembly means your process is generating rework. That's internal failure cost climbing. - **Failure Pareto analysis** ranks which measurements fail most often. This tells you where to invest prevention dollars. If 60% of failures are solder-related, that's a clear signal to improve your reflow profile or paste deposition. - **Yield trends over time** reveal whether quality is improving or degrading. A slow downward trend in FPY is COPQ increasing before anyone notices in the financial reports. - **Unit traceability** connects field failures back to test data. When a customer returns a unit, you can pull its complete test history and see whether it passed marginally or had anomalies that were within spec but near the edge. ## Reducing COPQ in Practice The path from high COPQ to low COPQ follows a predictable pattern: 1. **Instrument your process.** Add tests at each major manufacturing step. Upload everything to TofuPilot with consistent serial numbers and measurement names. 2. **Identify the biggest losses.** Use failure Pareto to find the top 3 failure modes. These typically account for 60-80% of your internal failures. 3. **Push detection upstream.** If final test catches a defect, ask whether an earlier test could have caught it before more value was added to the unit. 4. **Tighten limits proactively.** Use Cpk data from TofuPilot to identify measurements that are technically passing but drifting toward limits. Tightening process controls before failures occur is prevention, the cheapest category. 5. **Measure the result.** Track FPY improvement over time. Every percentage point of FPY improvement translates directly to lower scrap, less rework, and fewer field failures. ### Test Traceability for Aerospace (AS9100) URL: https://www.tofupilot.com/guides/test-traceability-for-aerospace-as9100-with-tofupilot Map AS9100 quality management requirements to TofuPilot features for test records, serial tracking, measurement history, and audit-ready exports. AS9100 requires aerospace manufacturers to maintain traceable, tamper-evident test records for every unit produced. TofuPilot maps directly to these requirements, giving you structured test data, serial number tracking, and exportable records without building a custom QMS. ## AS9100 Requirements for Test Records AS9100 Rev D builds on ISO 9001 with aerospace-specific additions. The clauses most relevant to test and inspection are: | AS9100 Clause | Requirement | TofuPilot Feature | |---|---|---| | 7.1.5 Monitoring and measuring resources | Calibrated equipment, traceable measurements | Measurements with units and limits, station identifiers | | 7.5.3 Control of documented information | Records retained, protected, retrievable | Immutable test records, cloud storage with retention | | 8.5.2 Identification and traceability | Unique identification of product throughout production | Serial number tracking, unit history page | | 8.6 Release of products and services | Evidence of conformity to acceptance criteria | Pass/fail results with measurement limits | | 8.7 Control of nonconforming outputs | Documented nonconformances with disposition | Failed test runs with root cause measurements | | 10.2 Nonconformity and corrective action | Records of nonconformities and actions taken | Failure history per unit, measurement trends | ## Structuring Tests for AS9100 Compliance A well-structured OpenHTF test produces the data AS9100 auditors look for: identified DUT, traceable measurements with limits, and a clear pass/fail determination. ```python filename="test_aerospace_pcba.py" # Aerospace PCBA acceptance test with AS9100-aligned structure import openhtf as htf from openhtf.util import units from tofupilot.openhtf import TofuPilot @htf.measures( htf.Measurement("insulation_resistance") .in_range(minimum=100) .doc("Per IPC-9252, minimum 100 Mohm between adjacent nets"), htf.Measurement("hi_pot_leakage") .in_range(maximum=0.001) .with_units(units.AMPERE) .doc("Dielectric withstand per MIL-STD-202, Method 301"), ) def dielectric_test(test): test.measurements.insulation_resistance = 450.2 test.measurements.hi_pot_leakage = 0.00012 @htf.measures( htf.Measurement("supply_voltage") .in_range(minimum=27.5, maximum=28.5) .with_units(units.VOLT) .doc("28V nominal bus, AS6081 compliant power supply"), htf.Measurement("quiescent_current") .in_range(maximum=0.050) .with_units(units.AMPERE), htf.Measurement("output_signal_snr") .in_range(minimum=40), ) def functional_acceptance(test): test.measurements.supply_voltage = 28.01 test.measurements.quiescent_current = 0.0124 test.measurements.output_signal_snr = 52.3 @htf.measures( htf.Measurement("temperature_cycle_drift_pct") .in_range(maximum=0.5) .doc("Output drift after -40C to +85C cycle per RTCA DO-160"), ) def environmental_screening(test): test.measurements.temperature_cycle_drift_pct = 0.18 def main(): test = htf.Test( dielectric_test, functional_acceptance, environmental_screening, ) with TofuPilot(test): test.execute(test_start=lambda: "AE-PCB-2026-00847") if __name__ == "__main__": main() ``` Each measurement includes units (where a constant exists), limits, and documentation strings. TofuPilot stores all of this, making it retrievable during audits. ## Record Retention and Retrieval AS9100 clause 7.5.3 requires that quality records are retained for the period specified by the customer or regulatory authority. In aerospace, this often means 7 to 30 years depending on the program. TofuPilot stores test records immutably. Records can't be edited or deleted through the normal interface. You can search by serial number, procedure, station, date range, or pass/fail status. For audit preparation, export test records from TofuPilot's dashboard. Each export includes the serial number, procedure name, station, operator (if captured), all measurements with limits, and the overall result. ## Nonconformance Tracking When a unit fails a test, AS9100 requires documented nonconformance handling. TofuPilot captures this at the measurement level. ```python filename="test_nonconformance.py" # Test that captures detailed failure data for nonconformance records import openhtf as htf from openhtf.util import units from tofupilot.openhtf import TofuPilot @htf.measures( htf.Measurement("solder_joint_resistance") .in_range(maximum=0.05) .with_units(units.OHM) .doc("IPC-A-610 Class 3 solder joint acceptance"), htf.Measurement("component_placement_offset") .in_range(maximum=0.1) .with_units(units.MILLIMETRE) .doc("Component X/Y offset from nominal pad center"), htf.Measurement("visual_inspection_defects") .in_range(maximum=0) .doc("Count of IPC-A-610 Class 3 defects found"), ) def incoming_inspection(test): test.measurements.solder_joint_resistance = 0.08 # FAIL: exceeds 0.05 ohm test.measurements.component_placement_offset = 0.04 test.measurements.visual_inspection_defects = 0 def main(): test = htf.Test(incoming_inspection) with TofuPilot(test): test.execute(test_start=lambda: "AE-PCB-2026-00848") if __name__ == "__main__": main() ``` The failed measurement (solder joint resistance at 0.08 ohm, limit 0.05 ohm) is recorded with its exact value. In TofuPilot's dashboard, failed runs surface immediately. The unit's history page shows every test attempt, so you can track rework and retest cycles. ## Station and Equipment Identification AS9100 requires traceability of monitoring and measuring equipment. TofuPilot records the station identifier for every test run, linking results to specific test fixtures and equipment. Configure station names to match your equipment calibration records. This creates a direct link between test results and the calibrated instruments that produced them. ## Audit-Ready Exports TofuPilot's dashboard provides everything AS9100 auditors typically request: - **Unit history.** Complete test record for any serial number, showing every procedure, measurement, and result. - **Measurement trends.** Control charts and Cpk values for any measurement across production, demonstrating process stability. - **Failure analysis.** Failure Pareto charts showing which measurements fail most often, supporting corrective action (clause 10.2). - **Station records.** Test results filtered by station, supporting equipment traceability requirements. All of this data is available without writing custom reports or export scripts. ### How to Set Up Kiosk Mode for Test Stations URL: https://www.tofupilot.com/guides/how-to-set-up-kiosk-mode-for-test-stations Kiosk mode locks a test station to the operator UI. Learn how to configure Chrome kiosk mode, auto-start, and barcode scanner integration. # How to Set Up Kiosk Mode for Test Stations Kiosk mode locks a test station's display to the operator interface. The operator sees only the test UI. No address bar, no taskbar, no desktop, no way to accidentally close the browser or open other applications. This guide covers how to set up kiosk mode on Windows and Linux for manufacturing test stations. ## Why Kiosk Mode | Problem | Kiosk Mode Solves It | |---------|---------------------| | Operator closes the browser | Browser restarts automatically | | Operator navigates away from the test UI | Address bar is hidden | | Operator opens other applications | Taskbar is hidden, Alt+Tab is disabled | | Station boots to a desktop | Browser opens automatically on startup | | Display is hard to read at arm's length | Browser zoom is preset to 150% | Kiosk mode turns a general-purpose PC into a dedicated test terminal. ## Prerequisites - Chrome or Chromium installed on the station PC - TofuPilot streaming URL for the station - Admin access to the station PC ## Step 1: Launch Chrome in Kiosk Mode ### Windows Create a shortcut or batch script that launches Chrome in kiosk mode pointing to the TofuPilot streaming URL. ```bat filename="start_operator_ui.bat" @echo off REM Launch Chrome in kiosk mode for operator UI start "" "C:\Program Files\Google\Chrome\Application\chrome.exe" ^ --kiosk ^ --disable-pinch ^ --overscroll-history-navigation=0 ^ --noerrdialogs ^ --disable-translate ^ --no-first-run ^ --fast ^ --fast-start ^ --disable-features=TranslateUI ^ --disk-cache-dir=nul ^ "https://tofupilot.app/streaming/your-station-room-id" ``` ### Linux Create a shell script for kiosk mode on Linux stations. ```bash filename="start_operator_ui.sh" #!/bin/bash # Launch Chromium in kiosk mode for operator UI chromium-browser \ --kiosk \ --disable-pinch \ --overscroll-history-navigation=0 \ --noerrdialogs \ --disable-translate \ --no-first-run \ --disable-features=TranslateUI \ "https://tofupilot.app/streaming/your-station-room-id" ``` ## Step 2: Auto-Start on Boot ### Windows Place the batch script in the Startup folder: ``` C:\Users\\AppData\Roaming\Microsoft\Windows\Start Menu\Programs\Startup\ ``` Or create a scheduled task that runs the script at logon. ### Linux (systemd) Create a systemd service that starts the browser after the display server is ready. ```ini filename="/etc/systemd/system/operator-ui.service" [Unit] Description=Operator UI Kiosk After=graphical.target [Service] Type=simple User=operator Environment=DISPLAY=:0 ExecStart=/home/operator/start_operator_ui.sh Restart=on-failure RestartSec=5 [Install] WantedBy=graphical.target ``` Enable the service: ```bash filename="terminal" sudo systemctl enable operator-ui.service ``` The `Restart=on-failure` setting means the browser relaunches automatically if it crashes or gets closed. ## Step 3: Configure the Display | Setting | Value | How | |---------|-------|-----| | Browser zoom | 125-150% | Set in Chrome settings or via `--force-device-scale-factor=1.5` flag | | Screen timeout | Never | Disable sleep/screensaver in OS power settings | | Screen rotation | Landscape | Match the UI layout | | Resolution | Native | Don't downscale, the UI is responsive | ## Step 4: Set Up Barcode Scanner Most USB barcode scanners work as HID keyboard devices. When the operator scans a barcode, the characters are typed into the active input field. | Setting | Recommendation | |---------|---------------| | Scanner mode | USB HID Keyboard (default on most scanners) | | Suffix | Configure the scanner to send Enter after each scan | | Prefix | None (or configure to match your serial format) | | Focus | The TofuPilot operator UI auto-focuses the serial input field | The workflow becomes: operator scans barcode, Enter is sent automatically, test starts. No clicking, no typing. ## Step 5: Lock Down the Station ### Disable Keyboard Shortcuts (Windows) Use Group Policy or a third-party tool to disable: | Shortcut | Why | |----------|-----| | Alt+Tab | Prevents switching away from the operator UI | | Alt+F4 | Prevents closing the browser | | Ctrl+W | Prevents closing the tab | | Windows key | Prevents opening the Start menu | ### Disable Keyboard Shortcuts (Linux) Configure the window manager to ignore these shortcuts, or run Chrome in a minimal window manager (like Openbox) that doesn't bind them. ## Troubleshooting | Issue | Fix | |-------|-----| | Browser shows "No internet" on boot | The browser starts before the network is ready. Add a 10-second delay to the startup script, or use `Restart=on-failure` in systemd. | | Screen goes black after inactivity | Disable screen timeout in OS power settings and screensaver. | | Scanner types into the wrong field | The TofuPilot UI auto-focuses the serial input. If focus is lost, clicking anywhere on the page restores it. | | Browser updates break kiosk mode | Pin the Chrome/Chromium version or disable auto-updates on station PCs. | ### How to Build a Test Operator UI in Python URL: https://www.tofupilot.com/guides/how-to-build-a-test-operator-ui-in-python Build a production-ready operator interface for hardware tests using Python, OpenHTF, and TofuPilot. No frontend code required. # How to Build a Test Operator UI in Python Most test engineers start with terminal output. That works during development, but production operators need a dedicated screen with pass/fail indicators, prompts, and input fields. This guide shows how to build an operator UI using OpenHTF and TofuPilot without writing any frontend code. ## What the Operator Sees The finished operator interface runs in a browser and shows: - Serial number entry (barcode scanner or manual input) - Live phase progress as the test runs - Operator prompts with input fields (text, numbers, dropdowns, image-based choices) - Pass/fail result with color-coded display - Measurement values with limit status The operator never sees a terminal, a file browser, or Python code. ## Prerequisites - Python 3.10+ - OpenHTF installed (`pip install openhtf`) - TofuPilot Python SDK 1.11.0+ installed (`pip install tofupilot`) ## Step 1: Create Test Phases with Prompts Use OpenHTF's `user_input` plug to add operator interactions. Each prompt pauses the test until the operator responds. ```python filename="operator_ui_test.py" import openhtf as htf from openhtf.plugs import user_input from openhtf.util import units @htf.plug(prompts=user_input.UserInput) def phase_load_dut(test, prompts): """Wait for the operator to load the DUT into the fixture.""" prompts.prompt( "Place the board in the test fixture and close the clamp. " "Press Enter when ready." ) @htf.plug(prompts=user_input.UserInput) @htf.measures( htf.Measurement("label_present").with_args(docstring="Label inspection result"), ) def phase_visual_check(test, prompts): """Ask the operator to verify the label is present and correct.""" result = prompts.prompt( "Is the product label present and correctly aligned?", text_input=True, ) test.measurements.label_present = result ``` ## Step 2: Add Automated Measurements Mix operator prompts with automated instrument measurements. The operator sees both types of phases in the same interface. ```python filename="operator_ui_test.py" @htf.measures( htf.Measurement("supply_voltage_V") .in_range(minimum=4.9, maximum=5.1) .with_units(units.VOLT), htf.Measurement("boot_time_ms") .in_range(maximum=2000) .with_units(units.MILLISECOND), ) def phase_power_and_boot(test): """Automated: measure supply voltage and boot time.""" test.measurements.supply_voltage_V = 5.02 test.measurements.boot_time_ms = 1340 @htf.measures( htf.Measurement("wifi_rssi_dBm") .in_range(minimum=-70, maximum=-20) .with_units(units.DBM), ) def phase_wireless_check(test): """Automated: verify WiFi signal strength.""" test.measurements.wifi_rssi_dBm = -42 ``` ## Step 3: Add a Final Operator Step After automated tests complete, ask the operator to unload the DUT and apply a pass/fail sticker. ```python filename="operator_ui_test.py" @htf.plug(prompts=user_input.UserInput) def phase_unload(test, prompts): """Instruct the operator to remove the DUT and apply the label.""" prompts.prompt( "Remove the board from the fixture. " "Apply a PASS sticker if the test passed. " "Press Enter when done." ) ``` ## Step 4: Connect to TofuPilot and Run Wire all phases together and connect to TofuPilot. The operator UI streams automatically to a browser URL. ```python filename="operator_ui_test.py" from tofupilot.openhtf import TofuPilot test = htf.Test( phase_load_dut, phase_visual_check, phase_power_and_boot, phase_wireless_check, phase_unload, ) with TofuPilot(test): test.execute(test_start=lambda: input("Scan serial: ")) ``` When you run this script, TofuPilot prints a URL in the console. Open it in a browser on the operator's station. The operator sees each phase in sequence, responds to prompts, and gets a clear pass/fail result. ## Step 5: Configure the Station for Production | Setting | How | |---------|-----| | Kiosk mode | Launch Chrome with `--kiosk --app=` | | Auto-start | Add the command to the OS startup script | | Barcode scanner | Configure as USB HID keyboard (scan goes into the active input field) | | Large text | Set browser zoom to 150% | | Touchscreen | TofuPilot prompts work with touch input, no mouse needed | | Multiple stations | Each station gets its own streaming URL | ## Input Types Available TofuPilot's operator UI supports structured input types beyond simple text: | Input Type | Use Case | |-----------|----------| | Text | Serial numbers, operator notes | | Number | Manual measurements, counts | | Slider | Analog adjustments, subjective ratings | | Radio buttons | Choose one option from a list | | Checkbox | Confirm multiple items (checklist) | | Dropdown | Select from a long list of options | | Image choice | Pick from photos (component orientation, defect type) | | Toggle | Yes/no decisions | These inputs render automatically in the browser. The operator fills them in, and the values flow back to the test script as measurement data. ## Common Patterns ### Fail-Fast with Operator Notification ```python filename="operator_ui_test.py" from openhtf import PhaseResult @htf.measures( htf.Measurement("power_good").equals("PASS"), ) def phase_power_check(test): """Stop early if power-up fails. No point testing a dead board.""" result = "PASS" test.measurements.power_good = result if result != "PASS": return PhaseResult.STOP ``` When this phase fails, the operator sees the failure immediately in the browser with the failing measurement highlighted. They don't wait for the remaining phases to time out. ### Conditional Operator Steps ```python filename="operator_ui_test.py" @htf.plug(prompts=user_input.UserInput) @htf.measures( htf.Measurement("rework_action").with_args(docstring="Rework performed"), ) def phase_rework_prompt(test, prompts): """If a previous phase flagged an issue, ask operator to rework.""" action = prompts.prompt( "The solder inspection flagged pad U3. " "Rework the joint and press Enter when done.", text_input=True, ) test.measurements.rework_action = action ``` ### Open Source Test Executive Alternatives URL: https://www.tofupilot.com/guides/open-source-test-executive-alternatives Compare open source test executives for manufacturing: OpenHTF, OpenTAP, pytest, and HardPy. Features, operator UI, cost, and when to use each. # Open Source Test Executive Alternatives NI TestStand costs $3-5K per seat and runs only on Windows. Engineers are searching for open source alternatives that run on Linux, integrate with Git, and don't require proprietary licenses. This guide compares the main open source test executives for manufacturing: OpenHTF, OpenTAP, pytest (with hardware plugins), and HardPy. ## What a Test Executive Does A test executive manages the execution of test sequences. It handles: | Function | What It Does | |----------|-------------| | Sequencing | Runs test steps in order with branching and looping | | Measurement collection | Records values with units and limits | | Pass/fail decisions | Compares measurements to limits, determines result | | Operator interaction | Prompts for input, displays status | | Data logging | Stores results for traceability and analytics | | Plug/driver management | Initializes and tears down instrument connections | TestStand does all of this in a proprietary Windows application. The open source alternatives do it in code. ## Comparison Table | Feature | NI TestStand | OpenHTF | OpenTAP | pytest + plugins | HardPy | |---------|-------------|---------|---------|-----------------|--------| | Language | LabVIEW/C#/VB | Python | C#/.NET | Python | Python | | License | Proprietary ($3-5K) | Apache 2.0 | MPL 2.0 | MIT | Apache 2.0 | | Platform | Windows only | Windows/Linux/macOS | Windows/Linux | Windows/Linux/macOS | Windows/Linux/macOS | | Origin | National Instruments | Google | Keysight | Community | EverPin | | Measurements with limits | Yes | Yes | Yes | Via plugins | Yes | | Phase/step sequencing | Visual editor | Python decorators | C# classes | Test functions | Test functions | | Operator UI | Built-in OIs | Station Server | Operator Panel | None built-in | Browser panel | | Instrument plugins | NI drivers | Plugs system | DUT/Instrument plugins | Fixtures | Fixtures | | Data export | ATML, custom | JSON, protobuf | CSV, custom | JUnit XML | JSON | | Version control | Difficult (binary) | Git-native | Git-native | Git-native | Git-native | | Community | Large (NI forums) | Small (GitHub) | Small (forum) | Large (general pytest) | Tiny | ## OpenHTF OpenHTF is Google's open source test framework for manufacturing. It was built for testing consumer electronics at scale. ### Strengths - **Python-native.** Tests are Python functions decorated with measurements. No visual editor, no XML, no binary files. - **Plug system.** Instruments are injected into test phases automatically. Plugs handle initialization and teardown. - **Measurement validators.** Built-in support for ranges, tolerances, regex, and exact match. - **Station Server.** Web-based operator interface at `localhost:12000`. - **TofuPilot integration.** Full operator UI, data logging, and analytics via the TofuPilot SDK. ### Weaknesses - **Documentation.** Official docs from Google don't exist. The community (via openhtf.org) fills the gap. - **Community size.** ~640 GitHub stars, low Stack Overflow presence. - **Built-in UI.** The Station Server is functional but basic. Most production deployments use TofuPilot for the operator interface. ### Example ```python filename="openhtf_example.py" import openhtf as htf from openhtf.util import units @htf.measures( htf.Measurement("voltage_V") .in_range(minimum=4.9, maximum=5.1) .with_units(units.VOLT), ) def phase_voltage_check(test): """Measure and validate supply voltage.""" test.measurements.voltage_V = 5.01 test = htf.Test(phase_voltage_check) test.execute(test_start=lambda: input("Scan serial: ")) ``` ## OpenTAP OpenTAP is Keysight's open source test automation framework. It's written in C# and runs on .NET. ### Strengths - **Keysight backing.** Active development, commercial support available. - **Plugin ecosystem.** DUT, instrument, and result listener plugins. - **Test plan editor.** GUI for building test sequences without code. - **Cross-platform.** Runs on Windows and Linux via .NET. ### Weaknesses - **C#/.NET.** Most test engineers prefer Python. .NET adds deployment complexity on Linux. - **Operator panel.** Basic, requires Keysight Test Automation license for full features. - **Learning curve.** Plugin architecture is powerful but complex for simple tests. ### Example ```csharp filename="OpenTapExample.cs" [Display("Voltage Check", Description = "Measure supply voltage")] public class VoltageCheck : TestStep { [Display("Voltage")] [Unit("V")] public double Voltage { get; set; } public override void Run() { Voltage = 5.01; if (Voltage < 4.9 || Voltage > 5.1) UpgradeVerdict(Verdict.Fail); else UpgradeVerdict(Verdict.Pass); } } ``` ## pytest with Hardware Plugins pytest is the standard Python testing framework. With the right plugins, it can work for hardware testing. ### Strengths - **Massive community.** Most Python developers already know pytest. - **Fixture system.** Handles instrument setup/teardown cleanly. - **Plugin ecosystem.** Thousands of plugins for reporting, parallel execution, and CI integration. - **Low learning curve.** If you know Python, you know pytest. ### Weaknesses - **Not built for manufacturing.** No built-in measurement system with limits, units, or validators. - **No operator UI.** No prompts, no serial number input, no pass/fail display. - **No native data logging.** Requires custom code or third-party tools to store results. - **Designed for software testing.** Concepts like fixtures, marks, and parametrize don't map cleanly to hardware test sequences. ### Example ```python filename="test_hardware.py" def test_voltage_check(dmm_fixture): """Measure and validate supply voltage.""" voltage = dmm_fixture.measure_voltage() assert 4.9 <= voltage <= 5.1, f"Voltage {voltage}V out of range" ``` ## HardPy HardPy adds a browser-based operator panel to pytest for hardware testing. ### Strengths - **pytest-native.** Tests are regular pytest functions with hardware-specific additions. - **Browser UI.** Operator panel with test hierarchy, dialogs, and real-time charts. - **Measurement storage.** Built-in CouchDB integration for test results. - **Modern stack.** React frontend, clean API. ### Weaknesses - **New project.** Limited adoption, small community. - **Basic features.** No measurement limit system, no marginal bands, no analytics. - **CouchDB dependency.** Adds infrastructure complexity. ## When to Use Each | Situation | Recommendation | |-----------|---------------| | Starting a new production test system in Python | OpenHTF + TofuPilot | | Already using pytest for hardware tests | Add TofuPilot SDK for data logging and operator UI | | .NET shop with Keysight instruments | OpenTAP | | Need the simplest possible operator panel | HardPy | | Migrating from TestStand, want minimal change | OpenTAP (closest architecture) | | Migrating from TestStand, want Python | OpenHTF + TofuPilot | | Budget is zero, need it working today | OpenHTF with built-in Station Server | | Need analytics, traceability, and operator UI | Any framework + TofuPilot | ## Cost Comparison | Solution | Software License | Per-Station Cost | Operator UI | Data Analytics | |----------|-----------------|-----------------|-------------|---------------| | NI TestStand | $3-5K/seat | $3-5K | Included | NI InsightCM (extra) | | OpenHTF + TofuPilot | Free | $0 | Included (all tiers) | Included | | OpenTAP | Free (core) | $0 | Plugin (may need license) | Custom | | pytest + TofuPilot | Free | $0 | Via TofuPilot | Included | | HardPy | Free | $0 | Included | Basic | The open source alternatives eliminate per-seat licensing. The total cost of ownership depends on engineering time to set up and maintain the system. Pre-built integrations (like TofuPilot's OpenHTF and pytest support) reduce that cost significantly. ### Test Data Visualization for Hardware Teams URL: https://www.tofupilot.com/guides/test-data-visualization-for-hardware-teams Learn how to build dashboards and visualizations for hardware test data using TofuPilot's built-in analytics and charting tools. # Test Data Visualization for Hardware Teams Numbers in a spreadsheet don't show trends. Charts do. TofuPilot turns your raw test measurements into interactive dashboards that surface yield problems, measurement drift, and station issues at a glance. ## Why Visualize Test Data Hardware test data has three properties that make visualization essential: 1. **Volume.** A production line running 500 units/day across 5 test procedures generates 2,500 test runs daily. You can't read that in a table. 2. **Patterns.** Drift, clustering, bimodal distributions, and outliers are invisible in raw numbers but obvious in a chart. 3. **Context.** A measurement of 3.31V means nothing alone. Plotted against the last 10,000 readings with spec limits overlaid, it tells a story. ## Dashboard Views in TofuPilot ### Procedure Dashboard Every test procedure in TofuPilot gets an automatic dashboard showing: | Widget | What it shows | |--------|--------------| | FPY trend | First-pass yield over time (daily, weekly, monthly) | | Run history | Recent runs with pass/fail status and timing | | Failure pareto | Top failure modes ranked by frequency | | Measurement histograms | Distribution of each measurement across all runs | | Measurement trends | Each measurement plotted over time with limits | No setup required. Upload test data and the dashboards populate automatically. ### Unit History Search by serial number to see every test a unit has been through. This view shows: - All test procedures the unit has completed - Pass/fail status for each procedure - Full measurement data for every run - Timeline of when each test was executed ### Station Comparison Compare performance across test stations. This is the fastest way to find station-specific issues: - FPY by station - Measurement distributions by station - Cycle time by station - Failure modes by station If Station 3 has 5% lower yield than the others, the station comparison view makes it obvious. ## Building Effective Test Dashboards ### Start with FPY First-pass yield is the single most important metric for a production test operation. It tells you what percentage of units pass all tests on the first attempt. Track FPY at three levels: 1. **Overall FPY** across all procedures. This is your headline number. 2. **FPY per procedure.** Which test is causing the most failures? 3. **FPY per station.** Is one station dragging down the overall number? ### Add Measurement Distributions For each critical measurement, check the histogram. A healthy process looks like a tight bell curve centered well within the spec limits. Warning signs in the histogram: - **Skewed distribution**: Process is biased toward one limit - **Wide distribution**: High variance, process not well controlled - **Bimodal distribution**: Two peaks, suggesting mixed populations (different component lots, different operators, different stations) - **Flat distribution**: No central tendency, process is essentially random within limits ### Use Trend Charts for Drift Detection Plot critical measurements over time. The trend chart shows each individual reading as a data point, with spec limits drawn as horizontal lines. Look for: - **Gradual slope**: Measurement drifting toward a limit (fixture wear, calibration drift) - **Step change**: Sudden shift in the baseline (new component lot, process change) - **Increasing scatter**: Variance growing over time (loss of process control) - **Periodic pattern**: Oscillation tied to time of day or production cycle ### Failure Pareto The failure pareto chart ranks failure modes by frequency. Focus improvement efforts on the top bar. Fixing the #1 failure mode improves overall yield more than fixing #3, #4, and #5 combined (usually). ## Code: Exporting Data for Custom Visualizations TofuPilot's built-in dashboards cover most needs. For custom analysis, export data via the API. ```python filename="export_measurements.py" from tofupilot import TofuPilotClient import csv client = TofuPilotClient() runs = client.get_runs( procedure_id="MOTOR-PERFORMANCE", limit=1000, ) with open("motor_data.csv", "w", newline="") as f: writer = csv.writer(f) writer.writerow(["serial", "date", "torque_nm", "current_a", "status"]) for run in runs: for step in run.get("steps", []): for m in step.get("measurements", []): if m["name"] == "peak_torque": writer.writerow([ run["unit_under_test"]["serial_number"], run["created_at"], m["value"], None, "pass" if run["run_passed"] else "fail", ]) ``` ## Dashboard Anti-Patterns | Anti-pattern | Why it's bad | What to do instead | |-------------|-------------|-------------------| | Tracking only pass/fail | Misses drift and anomalies | Track individual measurements | | Weekly summary reports | Too slow to catch issues | Use live dashboards | | Separate dashboards per station | Can't compare across stations | Use station comparison view | | No spec limits on charts | Can't assess margin | Always overlay limits | | Too many metrics on one screen | Information overload | Focus on FPY + top 5 measurements | ### Analyze Test Step Performance URL: https://www.tofupilot.com/guides/how-to-analyze-test-step-performance-with-tofupilot Learn how to break down test performance by step to find which phases fail most, run slowest, or have the lowest Cpk using TofuPilot's step analysis. A test that fails tells you something went wrong. Step-level analysis tells you exactly where. TofuPilot breaks down every run by phase so you can pinpoint which steps fail most, which take longest, and which measurements are closest to their limits. ## Why Step-Level Analysis Matters Overall pass/fail rates hide problems. A test with 95% FPY sounds fine until you discover one phase accounts for 80% of all failures. Without step-level data, you're guessing where to focus improvement efforts. Step analysis answers three questions: Which phases have the lowest pass rate? Which measurements have the lowest Cpk? Which steps take the most time? ## Structuring Tests for Good Step Data Each OpenHTF phase becomes a step in TofuPilot. Name your phases clearly and keep each one focused on a single functional area. This makes the analysis meaningful. ```python filename="multi_phase_test.py" import openhtf as htf from openhtf.util import units from tofupilot.openhtf import TofuPilot @htf.measures( htf.Measurement("input_voltage") .with_units(units.VOLT) .in_range(minimum=11.5, maximum=12.5), htf.Measurement("input_current") .with_units(units.AMPERE) .in_range(maximum=2.0), ) def power_input_check(test): """Verify power supply input within spec.""" test.measurements.input_voltage = 12.03 test.measurements.input_current = 1.45 @htf.measures( htf.Measurement("reg_3v3_output") .with_units(units.VOLT) .in_range(minimum=3.25, maximum=3.35), htf.Measurement("reg_5v_output") .with_units(units.VOLT) .in_range(minimum=4.90, maximum=5.10), htf.Measurement("reg_ripple") .in_range(maximum=50.0), ) def voltage_regulator_test(test): """Test all onboard voltage regulators.""" test.measurements.reg_3v3_output = 3.30 test.measurements.reg_5v_output = 4.98 test.measurements.reg_ripple = 22.4 @htf.measures( htf.Measurement("i2c_ack_received").with_allowed_values(True), htf.Measurement("spi_loopback_ok").with_allowed_values(True), htf.Measurement("uart_baud_error_pct") .in_range(maximum=2.0), ) def communication_bus_test(test): """Verify I2C, SPI, and UART interfaces.""" test.measurements.i2c_ack_received = True test.measurements.spi_loopback_ok = True test.measurements.uart_baud_error_pct = 0.8 @htf.measures( htf.Measurement("wifi_rssi") .in_range(minimum=-70, maximum=-20), htf.Measurement("bluetooth_pair_time") .in_range(maximum=5.0), ) def wireless_connectivity_test(test): """Test WiFi and Bluetooth connections.""" test.measurements.wifi_rssi = -45 test.measurements.bluetooth_pair_time = 2.3 @htf.measures( htf.Measurement("flash_write_speed") .in_range(minimum=10.0), htf.Measurement("flash_read_speed") .in_range(minimum=20.0), htf.Measurement("eeprom_verify_ok").with_allowed_values(True), ) def memory_test(test): """Verify flash and EEPROM read/write performance.""" test.measurements.flash_write_speed = 14.2 test.measurements.flash_read_speed = 28.7 test.measurements.eeprom_verify_ok = True def main(): test = htf.Test( power_input_check, voltage_regulator_test, communication_bus_test, wireless_connectivity_test, memory_test, ) with TofuPilot(test): test.execute(test_start=lambda: "BOARD-042") if __name__ == "__main__": main() ``` This test has five distinct phases, each covering a different subsystem. TofuPilot tracks pass rates, measurement distributions, and timing for each one independently. ## Finding Your Weakest Steps In TofuPilot's step analysis view, sort phases by pass rate to find the weakest links. A phase with 92% pass rate in a five-phase test dominates your overall yield loss. Look at the measurements within that phase. One measurement with a Cpk below 1.0 means your process variation is too wide for the spec. Either tighten the process or widen the spec if the product can tolerate it. ## Identifying Slow Steps Step timing data reveals bottlenecks. If your wireless connectivity test takes 8 seconds out of a 12-second total, that's where to focus cycle time reduction. Common fixes for slow steps: reduce retry counts, parallelize independent measurements, or optimize instrument communication (batch queries instead of sequential reads). ## Using Step Data to Prioritize Combine pass rate, Cpk, and timing into a prioritized list. A step that fails often and runs slowly is the highest-impact target. A step with high Cpk and fast execution needs no attention. TofuPilot's measurement analytics shows all of this per phase. Review it weekly to track whether your process improvements are actually moving the numbers. ### Run-over-Run Test Comparison with TofuPilot URL: https://www.tofupilot.com/guides/run-over-run-test-comparison-with-tofupilot Learn how to compare hardware test runs side by side in TofuPilot to diagnose failures and track measurement changes across units. # Run-over-Run Test Comparison with TofuPilot When a unit fails, the first question is always: "What's different from a passing unit?" Run-over-run comparison in TofuPilot lets you put two or more test runs side by side and see exactly where they diverge. ## When to Use Run Comparison - **Diagnosing a failure**: Compare a failed run to a recent passing run for the same procedure - **Investigating a retest**: Compare first test to retest for the same unit - **Validating a fix**: Compare runs before and after a corrective action - **Tracking unit history**: Compare the same unit's results across different test stages - **Benchmarking stations**: Compare the same unit tested on different stations ## How Run Comparison Works TofuPilot stores every measurement from every run. When you compare runs, the system aligns measurements by name and shows the values side by side with their limits. ``` Run A (Pass) Run B (Fail) ──────────── ──────────── vcc_3v3 3.30 V ✓ 3.28 V ✓ vcc_1v8 1.81 V ✓ 1.74 V ✗ clk_freq 24.00 MHz ✓ 23.98 MHz ✓ boot_time 320 ms ✓ 1,240 ms ✗ current_idle 45 mA ✓ 78 mA ✗ ``` Three measurements differ significantly. The 1.8V rail is low, boot time is 4x longer, and idle current is 73% higher. These symptoms together point to a partial short on the 1.8V power rail causing excess current draw and slow boot. ## Comparing Runs in TofuPilot ### Step 1: Find the Runs Navigate to the procedure page and filter to find the runs you want to compare. Common filters: | Filter | Use case | |--------|----------| | Status: Failed | Find failing runs to diagnose | | Serial number | Find all runs for a specific unit | | Date range | Narrow to a time period | | Station | Compare across stations | ### Step 2: Select Runs for Comparison Select two or more runs from the run list. TofuPilot aligns their measurements by step name and measurement name. ### Step 3: Read the Comparison Focus on measurements where the values differ significantly. Small variations (3.30V vs. 3.31V) are normal measurement noise. Large deviations (1.81V vs. 1.74V) indicate a real difference. Color coding helps: - **Green**: Both values within limits - **Red**: Value outside limits - **Yellow**: Value within limits but significantly different from the reference ## Common Comparison Patterns ### Pattern 1: Single Measurement Failure One measurement fails, everything else is identical. This usually means: - Component value out of tolerance - Solder defect on that specific circuit - Test probe contact issue (retest to confirm) ### Pattern 2: Correlated Failures Multiple related measurements fail together (e.g., voltage low + current high + boot slow). This points to a systemic issue: - Power rail problem affecting multiple circuits - Firmware crash causing downstream test failures - Fixture contact issue on a shared connection ### Pattern 3: All Measurements Shifted Every measurement is slightly different from the reference, but most are still within limits. This suggests: - Different environmental conditions (temperature affecting all measurements) - Different station (instrument calibration differences) - Different component lot (systematic parameter shift) ### Pattern 4: Intermittent Failure Same unit, same station, same procedure. Sometimes passes, sometimes fails. Compare the passing and failing runs: - If the failing measurement is always the same one, it's a marginal value near a limit - If different measurements fail each time, it's likely a contact issue (pogo pin, cable) - If the pattern is time-dependent, check for thermal effects ## Comparing Across Production Batches Run comparison isn't just for debugging. Use it to validate that a new production batch matches the previous one. 1. Select a representative passing run from batch N 2. Select the first runs from batch N+1 3. Compare measurement distributions If batch N+1 measurements are systematically shifted (even if still within limits), investigate before the full batch runs through production. ## Using the API for Programmatic Comparison ```python filename="compare_runs.py" from tofupilot import TofuPilotClient client = TofuPilotClient() # Get two runs to compare run_pass = client.get_run(run_id="run-id-pass") run_fail = client.get_run(run_id="run-id-fail") # Compare measurements for step_p, step_f in zip(run_pass["steps"], run_fail["steps"]): for m_p, m_f in zip(step_p["measurements"], step_f["measurements"]): diff = abs(m_p["value"] - m_f["value"]) if diff > 0: pct = diff / m_p["value"] * 100 if m_p["value"] != 0 else float("inf") status = "DIFF" if pct > 5 else "ok" print(f"{m_p['name']:30s} {m_p['value']:10.3f} {m_f['value']:10.3f} {pct:6.1f}% {status}") ``` This script highlights measurements that differ by more than 5%, giving you a quick programmatic way to identify where two runs diverge. ### Root Cause Analysis for Hardware Test Failures URL: https://www.tofupilot.com/guides/root-cause-analysis-for-hardware-test-failures Learn how to use TofuPilot's test data to trace hardware failures back to their root cause using measurement trends and run comparisons. # Root Cause Analysis for Hardware Test Failures A test fails. The question is never just "what failed" but "why did it fail, and will it fail again?" TofuPilot stores every measurement from every run, so you can trace failures back to their source instead of guessing. ## The Root Cause Analysis Problem in Hardware Software bugs leave stack traces. Hardware failures leave measurements. The difference: a stack trace points to one line of code, but an out-of-spec voltage reading could mean a bad solder joint, a drifting power supply, a faulty test fixture, or a component lot issue. Root cause analysis in hardware means correlating test data across multiple dimensions: time, station, component lot, operator, and environmental conditions. ## Step 1: Identify the Failure Pattern Start by filtering failed runs in TofuPilot's dashboard. | Filter | Purpose | |--------|---------| | Procedure | Narrow to the specific test | | Date range | Find when failures started | | Station | Check if failures cluster on one station | | Status: Failed | See only failed runs | Look for clustering. If failures concentrate on one station, one shift, or one date range, you've already narrowed the search. ## Step 2: Compare Failing vs. Passing Runs TofuPilot lets you compare runs side by side. Select a failing run and a passing run from the same procedure, then compare their measurements. ``` Passing run (UNIT-3021): Failing run (UNIT-3045): ┌─────────────────────────┐ ┌─────────────────────────┐ │ vcc_3v3: 3.30 V ✓ │ │ vcc_3v3: 3.28 V ✓ │ │ vcc_1v8: 1.81 V ✓ │ │ vcc_1v8: 1.76 V ✗ │ │ clk_freq: 24.00 MHz ✓ │ │ clk_freq: 23.95 MHz ✓ │ │ current: 45 mA ✓ │ │ current: 62 mA ✗ │ └─────────────────────────┘ └─────────────────────────┘ ``` In this example, the 1.8V rail is low and current draw is high. These two symptoms together point to a short or partial short on the 1.8V rail. ## Step 3: Check the Measurement Trend A single comparison shows you the "what." The trend shows you the "when." Open the measurement timeline for `vcc_1v8` across all runs. If the 1.8V reading was stable at 1.81V for weeks and then started dropping on March 3rd, something changed on March 3rd. Check: - Was a new component lot introduced? - Was the test fixture serviced? - Did the station's power supply get recalibrated? ## Step 4: Correlate with Component Lots TofuPilot tracks unit metadata including component information. If failures cluster around units built with a specific component lot, you've found a supplier quality issue. ```python filename="lot_analysis.py" from tofupilot import TofuPilotClient client = TofuPilotClient() # Get all failed runs for a specific procedure runs = client.get_runs( procedure_id="BOARD-FUNCTIONAL", run_passed=False, limit=100, ) # Check if failures correlate with component lots lot_counts = {} for run in runs: lot = run.get("unit_under_test", {}).get("batch", "unknown") lot_counts[lot] = lot_counts.get(lot, 0) + 1 for lot, count in sorted(lot_counts.items(), key=lambda x: -x[1]): print(f"Lot {lot}: {count} failures") ``` ## Step 5: Isolate the Variable Root cause analysis is a process of elimination. TofuPilot helps you hold variables constant while changing one at a time: | Variable | How to control it | |----------|-------------------| | Station | Filter by station ID | | Operator | Filter by shift/time | | Component lot | Filter by batch/lot | | Fixture | Check station metadata | | Environment | Compare with temperature logs | When you can reproduce the failure by controlling one variable (e.g., "all failures are from Station 3"), you've isolated the root cause. ## Common Root Cause Patterns ### Station-Specific Failures Failures cluster on one test station. Usually caused by: - Worn pogo pins or test probes - Loose cable connections - Calibration drift on station instruments ### Lot-Specific Failures Failures cluster around units from a specific component lot. Usually caused by: - Supplier quality escape - Component parameter shift - Wrong component revision ### Time-Correlated Failures Failures start at a specific date and continue. Usually caused by: - Process change (solder profile, firmware version) - Environmental change (humidity, temperature) - Fixture wear reaching a threshold ### Intermittent Failures Failures appear randomly across stations and lots. Usually caused by: - Marginal design (values close to limits) - Test measurement noise - Environmental sensitivity ## From Root Cause to Corrective Action Once you've identified the root cause, TofuPilot's data helps you verify the fix. Run the same tests after the corrective action and compare the measurement distributions before and after. If the 1.8V rail readings shift back to 1.81V and current draw returns to normal, your fix worked. Track the corrective action's effectiveness over time. TofuPilot's trend views show whether the fix holds or if the problem returns. ### Manufacturing Test Frameworks Compared URL: https://www.tofupilot.com/guides/manufacturing-test-frameworks-compared-openhtf-vs-pytest-vs-opentap-vs-teststand A comparison of manufacturing test frameworks (OpenHTF, pytest, OpenTAP, TestStand) with code examples, feature matrices, cost analysis, and guidance on. Four frameworks dominate manufacturing test automation: OpenHTF, pytest, OpenTAP, and NI TestStand. Each makes different tradeoffs between structure, flexibility, cost, and ecosystem. This guide compares them with real code, concrete metrics, and decision criteria so you can pick the right one. ## Framework Overview | Framework | Language | License | Origin | Focus | |-----------|---------|---------|--------|-------| | **OpenHTF** | Python | Apache 2.0 (free) | Google | Manufacturing/production test | | **pytest** | Python | MIT (free) | Community | Software testing, adapted for hardware | | **OpenTAP** | C# / Python | MPL 2.0 (free) | Keysight | Instrument-heavy test automation | | **NI TestStand** | LabVIEW / C / Python | Commercial ($4,310/seat) | NI (Emerson) | Enterprise manufacturing test | ## Feature Comparison Matrix | Feature | OpenHTF | pytest | OpenTAP | TestStand | |---------|---------|--------|---------|-----------| | **Structured measurements** | Built-in (name, value, limits, units) | Manual (assert only) | Plugin-based | Built-in | | **Serial number input** | Built-in prompt | Manual | Plugin | Built-in | | **Operator UI** | Built-in web UI | None | Built-in (GUI editor) | Built-in (Sequence Editor) | | **Test sequencing** | Phase ordering | Function ordering (plugins) | Step ordering (GUI) | Sequence files (GUI) | | **Parallel DUT** | Limited | Native (pytest-xdist) | Native | Native | | **Instrument drivers** | Plugs (Python) | Fixtures (Python) | Plugins (C#/Python) | NI drivers (LabVIEW) | | **Report format** | Protobuf | JUnit XML | XML/JSON | XML/database | | **Version control** | Git (Python files) | Git (Python files) | Git (XML + code) | Difficult (binary .seq files) | | **CI/CD integration** | Native (Python) | Native (Python) | Possible | Difficult | | **Cross-platform** | Linux, macOS, Windows | Linux, macOS, Windows | Linux, Windows | Windows only | | **Community** | Small (~640 stars) | Massive (11K+ stars) | Small (~200 stars) | Large (NI forums) | | **Learning curve** | Medium | Low | Medium | High | | **Annual cost (5 seats)** | $0 | $0 | $0 | ~$21,550 | ## The Same Test in Each Framework A simple functional test: measure a 3.3V rail voltage, check it's within 3.2V to 3.4V. ### OpenHTF ```python filename="comparison/openhtf_example.py" import openhtf as htf from openhtf.util import units from tofupilot.openhtf import TofuPilot class DutPlug(htf.plugs.BasePlug): def setUp(self): self.voltage = 3.31 # Replace with instrument read def tearDown(self): pass @htf.measures( htf.Measurement("rail_3v3") .in_range(3.2, 3.4) .with_units(units.VOLT), ) @htf.plug(dut=DutPlug) def test_power(test, dut): test.measurements.rail_3v3 = dut.voltage def main(): test = htf.Test(test_power, procedure_id="FCT-001", part_number="PCBA-100") with TofuPilot(test): test.execute(test_start=lambda: input("Serial: ")) if __name__ == "__main__": main() ``` **Strengths:** Measurements are structured data (name, value, limits, units). One line for TofuPilot integration. Operator gets a serial number prompt automatically. ### pytest ```python filename="comparison/pytest_example.py" import pytest from tofupilot import TofuPilotClient @pytest.fixture def dut(): connection = {"voltage": 3.31} # Replace with real connection yield connection def test_power_rail(dut): voltage = dut["voltage"] assert 3.2 <= voltage <= 3.4, f"3.3V rail: {voltage}V" ``` **Strengths:** Familiar to every Python developer. Huge plugin ecosystem. Flexible fixture system. Great for R&D and validation. **Weakness:** Measurements are implicit (assert statements). No structured data for analytics without extra code. ### OpenTAP ```csharp filename="comparison/opentap_example.cs" // OpenTAP C# test step using OpenTap; [Display("Power Rail Test", Group: "FCT")] public class PowerRailStep : TestStep { [Display("Lower Limit")] public double LowerLimit { get; set; } = 3.2; [Display("Upper Limit")] public double UpperLimit { get; set; } = 3.4; public override void Run() { double voltage = 3.31; // Replace with instrument read Results.Publish("rail_3v3", new { Voltage = voltage }); if (voltage < LowerLimit || voltage > UpperLimit) UpgradeVerdict(Verdict.Fail); } } ``` **Strengths:** GUI step editor for non-programmers. Strong Keysight instrument integration. Plugin architecture for test plans. **Weakness:** C# primary language (Python plugin exists but is secondary). Smaller community. ### NI TestStand TestStand uses a visual sequence editor. The equivalent test is a sequence file (.seq) with a "Numeric Limit Test" step configured via GUI: test value = voltage reading, low limit = 3.2, high limit = 3.4. **Strengths:** Mature, enterprise-grade. Deep NI hardware integration. Built-in report generation, database logging, parallel execution. **Weakness:** $4,310/seat/year. Windows only. Binary sequence files don't version-control well. Tied to NI ecosystem. ## Cost Analysis | | OpenHTF | pytest | OpenTAP | TestStand | |--|---------|--------|---------|-----------| | License (5 seats) | $0 | $0 | $0 | $21,550/year | | License (20 seats) | $0 | $0 | $0 | $86,200/year | | Runtime deployment | Free | Free | Free | Additional runtime licenses | | Training | Self-taught (docs) | Self-taught (docs) | Self-taught (docs) | NI training courses ($2K+) | | Vendor lock-in | None | None | Low (Keysight-adjacent) | High (NI ecosystem) | | Support | Community (GitHub) | Community (massive) | Community + Keysight | NI support contract | ## When to Use Each Framework | Scenario | Best Choice | Why | |----------|------------|-----| | Production FCT, Python team | **OpenHTF** | Built for manufacturing test, structured measurements, operator UI | | R&D validation, firmware CI | **pytest** | Flexible, fast iteration, CI/CD native, huge ecosystem | | Multi-vendor instruments, enterprise | **OpenTAP** | Plugin architecture, GUI editor, Keysight integration | | Existing NI hardware, large enterprise | **TestStand** | Deep NI integration, enterprise support, existing infrastructure | | Small team, budget-conscious | **OpenHTF** or **pytest** | Zero license cost, Python ecosystem | | Mixed (R&D + production) | **pytest** + **OpenHTF** | pytest for validation, OpenHTF for production | ## Migration Paths ### From TestStand to OpenHTF | TestStand Concept | OpenHTF Equivalent | |-------------------|-------------------| | Sequence file (.seq) | Python test script | | Step types | Phase functions | | Numeric Limit Test | `@htf.measures` with `.in_range()` | | String Value Test | `@htf.measures` with `.equals()` | | Step module (code) | Plug class | | Process model | TofuPilot integration | | Report generation | TofuPilot dashboard | | UUT serial number | `test.execute(test_start=lambda: input("Serial: "))` | ### From pytest to OpenHTF | pytest Concept | OpenHTF Equivalent | |---------------|-------------------| | Test function | Phase function | | Fixture | Plug | | Assert statement | `@htf.measures` with validators | | conftest.py | Plug classes in shared module | | JUnit XML | Protobuf output + TofuPilot | ## TofuPilot Integration All four frameworks work with TofuPilot: | Framework | Integration Method | Effort | |-----------|-------------------|--------| | OpenHTF | Native (`with TofuPilot(test)`) | 1 line | | pytest | Python SDK (`TofuPilotClient`) | ~10 lines per test | | OpenTAP | REST API or Python SDK | Medium | | TestStand | REST API | Medium | ## Decision Flowchart 1. **Do you have existing TestStand infrastructure?** Yes, and it works well: stay with TestStand. Yes, but you want to migrate: move to OpenHTF. 2. **Are you building production tests?** Yes: OpenHTF. It was built for this. 3. **Are you doing R&D or firmware validation?** Yes: pytest. More flexible, better CI/CD. 4. **Do you need a GUI for non-programmers?** Yes: OpenTAP. Visual step editor. 5. **Unsure?** Start with pytest. It's the easiest to learn and you can always add OpenHTF for production later. ### Collaborative Test Analysis with TofuPilot URL: https://www.tofupilot.com/guides/collaborative-test-analysis-with-tofupilot Learn how to share hardware test data across teams using TofuPilot's centralized platform for collaborative debugging and quality analysis. # Collaborative Test Analysis with TofuPilot Hardware debugging is a team sport. The test engineer sees the failure. The design engineer understands the circuit. The manufacturing engineer knows the process. But they're all looking at different data in different tools. TofuPilot puts everyone on the same page. ## The Collaboration Problem When a test fails, the investigation usually goes like this: 1. Test engineer sees the failure on the station PC 2. Test engineer screenshots the data or exports a CSV 3. Test engineer emails the CSV to the design engineer 4. Design engineer asks for more context ("What station? What lot? What were the other measurements?") 5. Test engineer goes back to the station, pulls more data, sends another email 6. Manufacturing engineer gets looped in, asks for different data 7. Repeat Every handoff loses context. Every email is a snapshot that's already outdated. The investigation takes days instead of hours. ## How TofuPilot Enables Collaboration ### Single Source of Truth Every test result from every station lives in TofuPilot. When someone asks "What happened to unit UNIT-5501?" everyone looks at the same data. No more "Which version of the spreadsheet are you looking at?" No more "Can you re-export with the timestamps included?" ### Shared Dashboards TofuPilot's dashboards are accessible to everyone on the team. The test engineer, design engineer, manufacturing engineer, and quality manager all see the same metrics, the same trends, the same failure paretos. Share a dashboard link instead of attaching a report. The recipient sees live data, can filter and drill down, and can explore on their own without asking the test engineer for help. ### Investigation Workflow When a quality issue surfaces, the collaborative investigation looks like this: 1. Quality engineer notices a yield drop on the TofuPilot dashboard 2. They filter to see which station, which time period, which failure mode 3. They share the filtered view link with the test engineer: "Station 3 started failing Power Rail Check at 2 PM" 4. Test engineer opens the link, sees the exact measurements, compares with passing runs 5. Design engineer opens the same link, recognizes the measurement pattern: "That 1.8V drop looks like a decoupling cap issue" 6. Manufacturing engineer checks the BOM: "New capacitor lot arrived this morning" Same data, different expertise, one platform. The issue is identified in 30 minutes, not 3 days. ## Team Roles in TofuPilot | Role | What they look at | How they use it | |------|------------------|----------------| | Test engineer | Individual run results, station status | Debug test failures, maintain stations | | Design engineer | Measurement trends, distributions | Identify design margin issues | | Manufacturing engineer | Yield trends, failure paretos | Process optimization, supplier issues | | Quality manager | FPY dashboards, compliance records | Release decisions, audit evidence | | Field engineer | Unit history by serial number | Diagnose field returns | Everyone uses the same platform but focuses on different views. ## Cross-Team Visibility ### Test to Design Design engineers often don't see production test data until something goes wrong. With TofuPilot, they can proactively monitor measurement distributions for their circuits. A design engineer who sees that their 3.3V rail measurements are clustering at 3.34V (near the 3.35V limit) can adjust the design or tighten the component spec before failures start. ### Test to Manufacturing Manufacturing engineers need to correlate test failures with process variables: which solder profile, which pick-and-place program, which component lot. TofuPilot's structured data makes these correlations possible without manual data alignment. ### Test to Field When a customer reports an issue, the field engineer searches by serial number and sees every test the unit ever passed. If the unit's production measurements were marginal, that's a likely root cause. If they were nominal, the issue is probably from field conditions or aging. ## Replacing Email-Based Debugging | Email workflow | TofuPilot workflow | |---------------|-------------------| | "Can you send me the data for UNIT-5501?" | Search by serial number | | "What's the yield been this week?" | Open the procedure dashboard | | "Which station is failing?" | Station comparison view | | "Here's my analysis (attached Excel)" | Share a filtered dashboard link | | "Can you re-export with more columns?" | The recipient filters the data themselves | Every email in the left column is a context switch and a delay. Every action in the right column takes seconds and is self-service. ## Getting Started 1. **Connect all stations**: Every test station pushes results to TofuPilot. No data silos. 2. **Invite the team**: Give access to everyone who touches test data: test, design, manufacturing, quality, and field teams. 3. **Share dashboard links**: Replace email attachments with live links. 4. **Build the habit**: When someone asks a test data question, the answer starts with "Open TofuPilot" instead of "Let me pull a report." The biggest impact isn't the tool itself. It's that everyone can answer their own questions about test data without waiting for someone else to extract it for them. ### What Is Adaptive Testing for Manufacturing URL: https://www.tofupilot.com/guides/what-is-adaptive-testing-for-manufacturing Adaptive testing uses real-time data to adjust test sequences, skip redundant checks, and reduce cycle time. Learn how it works and where it applies. # What Is Adaptive Testing for Manufacturing Adaptive testing adjusts the test sequence in real time based on data from earlier test steps, historical results, or process conditions. Instead of running every unit through every test, adaptive testing decides which tests to run, which to skip, and which limits to apply based on what the data says. This guide covers how adaptive testing works, the different levels of adaptation, and where it produces the most value. ## Static vs Adaptive Testing | Aspect | Static Testing | Adaptive Testing | |--------|---------------|-----------------| | Test sequence | Same for every unit | Adjusted per unit based on data | | Test time | Fixed | Variable (shorter for good units) | | Limits | Same for every unit | Can be tightened or relaxed based on context | | Decision logic | Pass/fail per measurement | Pass/fail plus risk score | | Data dependency | None (runs blind) | Requires historical data and inline analysis | In semiconductor testing, adaptive test has been deployed for over a decade. Advantest and Teradyne offer platforms that skip tests based on upstream wafer data, reducing test time by 10-50%. In discrete manufacturing, the concept is newer but the same principles apply. ## Levels of Adaptive Testing | Level | What Adapts | Example | Complexity | |-------|-----------|---------|-----------| | 1. Fail-fast | Test order | Run the highest-failure-rate test first, stop early on failure | Low | | 2. Skip on pass | Test coverage | If power-up passes, skip the detailed voltage rail test | Medium | | 3. Data-driven skip | Test selection | ML model predicts this unit will pass based on upstream data, skip test | High | | 4. Dynamic limits | Pass/fail criteria | Tighten limits for a batch from a new supplier, relax for a proven lot | High | | 5. Autonomous closure | Test termination | AI determines sufficient testing has been performed for this unit | Very high | Most manufacturing teams operate at level 1 (fail-fast ordering) without calling it adaptive testing. Levels 2-3 deliver the biggest cycle time reductions. Levels 4-5 are emerging capabilities. ## Where Adaptive Testing Produces Value | Scenario | Static Test Time | Adaptive Test Time | Savings | |----------|-----------------|-------------------|---------| | High-yield product (98% FPY) | 60 seconds | 45 seconds (skip redundant checks on passing units) | 25% | | Multi-variant product | 90 seconds (all tests) | 50-70 seconds (test only the variant-specific steps) | 22-44% | | Mature product, stable process | 60 seconds | 30 seconds (skip tests with 0% historical failure rate) | 50% | | New product, unstable process | 60 seconds | 60 seconds (don't skip anything, need the data) | 0% | Adaptive testing delivers the most value when yield is high and the product is mature. When you're still learning, run everything and collect data. ## Prerequisites - Python 3.10+ - OpenHTF installed (`pip install openhtf`) - TofuPilot Python SDK installed (`pip install tofupilot`) ## Step 1: Implement Fail-Fast Ordering The simplest form of adaptive testing: put the phases that fail most often first. When a unit fails early, skip the remaining phases. ```python filename="adaptive_test.py" import openhtf as htf from openhtf.util import units from openhtf import PhaseResult @htf.measures( htf.Measurement("power_good").equals("PASS"), ) def phase_power_check(test): """Highest failure rate test. Run first, stop on fail.""" result = "PASS" test.measurements.power_good = result if result != "PASS": return PhaseResult.STOP @htf.measures( htf.Measurement("firmware_version").equals("3.1.0"), ) def phase_firmware(test): """Second highest failure rate. Run next.""" test.measurements.firmware_version = "3.1.0" @htf.measures( htf.Measurement("output_voltage_V") .in_range(minimum=4.9, maximum=5.1) .with_units(units.VOLT), ) def phase_output(test): """Rarely fails. Run last.""" test.measurements.output_voltage_V = 5.01 ``` ## Step 2: Skip Redundant Tests If a test phase has had 0% failure rate for the last 10,000 units, consider removing it from the sequence. Use TofuPilot's failure Pareto to identify these phases. ```python filename="adaptive_test.py" @htf.measures( htf.Measurement("communication_check").equals("PASS"), ) def phase_communication(test): """0.01% failure rate. Candidate for skip on mature products.""" test.measurements.communication_check = "PASS" ``` The decision to skip should be data-driven. TofuPilot tracks failure rates per test step. Open the Analytics tab to see which phases have zero or near-zero failure rates across thousands of units. ## Step 3: Log Results for Continuous Learning Adaptive testing requires continuous data collection. Even when you skip a test on most units, run the full sequence on a sample to validate that the skipped tests are still passing. ```python filename="adaptive_test.py" from tofupilot.openhtf import TofuPilot test = htf.Test( phase_power_check, phase_firmware, phase_output, phase_communication, ) with TofuPilot(test): test.execute(test_start=lambda: input("Scan serial: ")) ``` ## Risks and Safeguards | Risk | Safeguard | |------|----------| | Skipping a test that would have caught a defect | Run full sequence on 5-10% of units as audit sample | | Process change invalidates skip decisions | Re-evaluate after any ECO, supplier change, or process change | | Over-optimization on stable data | Set a minimum test coverage floor that can't be reduced | | Regulatory requirements for 100% testing | Some industries (medical, aerospace) require every test on every unit | Adaptive testing is not about testing less. It's about testing smarter. The goal is to maintain the same defect detection rate with less time and cost. ### Robotics Test Data Management with TofuPilot URL: https://www.tofupilot.com/guides/robotics-test-data-management-with-tofupilot Learn how to manage test data for autonomous vehicles and robotics systems using TofuPilot's multi-stage test tracking and sensor data storage. # Robotics Test Data Management with TofuPilot Robots go through more test stages than almost any other product category. Board-level tests, motor characterization, sensor calibration, firmware validation, system integration, environmental screening, and field simulation. TofuPilot tracks all of it under one serial number. ## The Robotics Testing Challenge A typical autonomous robot has: - 3-5 PCBAs, each with its own functional test - 6-12 motors or actuators, each needing characterization - Multiple sensor suites (LiDAR, cameras, IMUs) requiring calibration - Firmware that needs validation at each revision - System-level integration tests - Environmental tests (thermal, vibration, IP rating) - Field simulation or HIL testing Each test stage generates data. Without a centralized system, that data lives in different tools, different formats, on different machines. When a robot fails in the field, tracing back to which test stage missed the issue is nearly impossible. ## Setting Up Multi-Stage Test Tracking ### Define Your Test Procedures Create a procedure in TofuPilot for each test stage in your manufacturing process. | Procedure ID | Stage | What it tests | |-------------|-------|--------------| | `PCBA-MOTOR-CTRL` | Board test | Motor controller PCBA functional test | | `MOTOR-CHAR` | Subassembly | Motor torque/speed characterization | | `IMU-CAL` | Sensor cal | IMU offset and sensitivity calibration | | `SYS-INTEGRATION` | System | Full robot integration checks | | `THERMAL-CYCLE` | Environmental | Thermal cycling qualification | | `HIL-NAV` | HIL | Navigation algorithm validation | ### Link Tests to the Same Unit Every test run references the robot's serial number. TofuPilot automatically links all runs for a given serial, creating a complete test history. ```python filename="motor_controller_test.py" from tofupilot import TofuPilotClient client = TofuPilotClient() # Board-level test for the motor controller client.create_run( procedure_id="PCBA-MOTOR-CTRL", unit_under_test={ "serial_number": "ROBO-2025-0142", "part_number": "MC-BOARD-R3", }, run_passed=True, steps=[{ "name": "H-Bridge Driver", "step_type": "measurement", "status": True, "measurements": [{ "name": "gate_drive_voltage", "value": 12.1, "unit": "V", "limit_low": 11.5, "limit_high": 12.5, }], }, { "name": "Current Sense", "step_type": "measurement", "status": True, "measurements": [{ "name": "current_sense_gain", "value": 50.2, "unit": "mV/A", "limit_low": 48.0, "limit_high": 52.0, }], }], ) ``` ```python filename="motor_characterization.py" # Motor characterization for the same robot client.create_run( procedure_id="MOTOR-CHAR", unit_under_test={ "serial_number": "ROBO-2025-0142", "part_number": "DRIVE-ASSEMBLY-R2", }, run_passed=True, steps=[{ "name": "Stall Torque", "step_type": "measurement", "status": True, "measurements": [{ "name": "stall_torque_nm", "value": 2.45, "unit": "Nm", "limit_low": 2.2, "limit_high": 2.8, }], }, { "name": "No-Load Speed", "step_type": "measurement", "status": True, "measurements": [{ "name": "no_load_rpm", "value": 5820, "unit": "RPM", "limit_low": 5500, "limit_high": 6100, }], }], ) ``` ### Track Sensor Calibration Data Sensor calibration produces arrays of data. TofuPilot handles multi-dimensional measurements natively. ```python filename="imu_calibration.py" import numpy as np # IMU calibration with multi-axis data client.create_run( procedure_id="IMU-CAL", unit_under_test={"serial_number": "ROBO-2025-0142"}, run_passed=True, steps=[{ "name": "Accelerometer Offset", "step_type": "measurement", "status": True, "measurements": [ {"name": "accel_offset_x", "value": 0.012, "unit": "g", "limit_low": -0.05, "limit_high": 0.05}, {"name": "accel_offset_y", "value": -0.008, "unit": "g", "limit_low": -0.05, "limit_high": 0.05}, {"name": "accel_offset_z", "value": 0.003, "unit": "g", "limit_low": -0.05, "limit_high": 0.05}, ], }, { "name": "Gyro Bias", "step_type": "measurement", "status": True, "measurements": [ {"name": "gyro_bias_x", "value": 0.15, "unit": "deg/s", "limit_low": -0.5, "limit_high": 0.5}, {"name": "gyro_bias_y", "value": -0.22, "unit": "deg/s", "limit_low": -0.5, "limit_high": 0.5}, {"name": "gyro_bias_z", "value": 0.08, "unit": "deg/s", "limit_low": -0.5, "limit_high": 0.5}, ], }], ) ``` ## Tracing Field Failures Back to Production When a robot fails in the field, search by serial number in TofuPilot. You'll see every test it went through: 1. Did the motor controller board pass its functional test? What were the exact measurements? 2. Was the motor characterization nominal, or was it marginal? 3. Were the IMU calibration offsets within spec? 4. Did the system integration test pass cleanly, or were there retests? This trace often reveals the root cause. A motor that passed characterization at the edge of its torque spec is more likely to fail under real-world loads. ## Fleet-Level Analytics For robotics companies deploying fleets, TofuPilot's analytics work across your entire production. - Compare calibration distributions across production batches - Track motor characterization trends over time - Identify which test stage catches the most defects - Correlate field failure rates with production test margins If robots from batch 47 have a higher field failure rate, pull their production test data and compare measurement distributions against batch 46. The difference in the data points to the root cause. ### Real-Time Test Monitoring with TofuPilot URL: https://www.tofupilot.com/guides/real-time-test-monitoring-with-tofupilot Learn how to monitor hardware test operations in real time using TofuPilot's live dashboards, yield tracking, and instant failure alerts. # Real-Time Test Monitoring with TofuPilot By the time you open last week's test report, the damage is done. Real-time monitoring means seeing test results as they happen, so you can react to problems in minutes instead of days. ## Why Real-Time Matters A test station starts failing at 10 AM. Without real-time monitoring, nobody notices until the quality engineer pulls the weekly report on Friday. That's 4.5 days of potentially bad units moving downstream. With TofuPilot's live dashboards, the yield drop shows up immediately. The engineer investigates at 10:15 AM, finds a worn pogo pin on Station 3, and fixes it by 11 AM. Three bad units instead of three hundred. ## Setting Up Live Monitoring ### Step 1: Connect Your Test Stations Every test station pushes results to TofuPilot as runs complete. The data appears on dashboards within seconds. ```python filename="station_with_monitoring.py" from tofupilot import TofuPilotClient client = TofuPilotClient() # Results appear on the dashboard immediately after this call client.create_run( procedure_id="FINAL-FUNCTIONAL", unit_under_test={"serial_number": "UNIT-9921"}, run_passed=True, steps=[{ "name": "Power On Self Test", "step_type": "measurement", "status": True, "measurements": [{ "name": "boot_time_ms", "value": 342, "unit": "ms", "limit_high": 500, }], }], ) ``` ### Step 2: Open the Live Dashboard The procedure dashboard in TofuPilot shows: - **Run feed**: Latest runs appear at the top as they come in. Green for pass, red for fail. - **Rolling FPY**: Yield calculated over a sliding window (last 50 runs, last 24 hours, etc.) - **Measurement trends**: Live-updating charts for each measurement parameter. Put this on a monitor near the production floor. When the feed turns red, everyone sees it. ### Step 3: Monitor Multiple Stations If you run multiple stations for the same procedure, the dashboard shows results from all of them. Color-code or filter by station to spot station-specific issues. | View | What it tells you | |------|------------------| | All stations combined | Overall line performance | | Single station filtered | Whether one station is underperforming | | Station comparison | Side-by-side FPY and measurement distributions | ## What to Monitor ### Yield (FPY) The first number to watch. Track it at three time scales: - **Hourly**: Catches sudden problems (fixture failure, bad component reel) - **Daily**: Shows shift-to-shift variation - **Weekly**: Reveals longer-term trends (gradual fixture wear, seasonal effects) ### Failure Modes When a run fails, TofuPilot records which step and measurement caused the failure. The failure pareto updates in real time, showing you the current top failure modes. If a new failure mode suddenly appears at the top of the pareto, something changed. Investigate immediately. ### Measurement Values Even for passing units, track the actual measured values. A measurement that's drifting toward its limit is a future failure waiting to happen. TofuPilot's measurement trend chart shows each reading as a dot, with spec limits drawn as horizontal lines. When dots start clustering near a limit line, it's time to act. ### Cycle Time Test cycle time tells you about station health. If a test that normally takes 45 seconds starts taking 90 seconds, something is wrong (instrument communication timeout, retry loop, slow fixture actuation). ## Responding to Real-Time Alerts When the dashboard shows a problem: 1. **Check the failure mode.** What measurement is failing? Is it one measurement or multiple? 2. **Check the station.** Is the problem on one station or all stations? One station means a fixture or instrument issue. All stations means a product or process issue. 3. **Check the timing.** When did it start? What changed at that time (new component lot, shift change, fixture service)? 4. **Act.** If it's station-specific, take the station offline and inspect. If it's product-wide, hold the batch and investigate the component or process change. ## Real-Time vs. Batch Monitoring | Aspect | Batch (weekly reports) | Real-time (TofuPilot) | |--------|----------------------|----------------------| | Detection time | Days to weeks | Minutes | | Units at risk | Hundreds to thousands | Single digits | | Data format | Static PDF/Excel | Interactive dashboard | | Drill-down ability | Limited (aggregated data) | Full (individual runs) | | Station comparison | Manual cross-reference | Automatic | ## Production Floor Setup For maximum visibility, set up a dedicated monitoring display: 1. Mount a screen visible from the test area 2. Open TofuPilot's procedure dashboard in a browser 3. Set the time filter to "last 24 hours" 4. Auto-refresh keeps the data current Operators and engineers can glance at the screen between test runs. When something goes wrong, the red indicators are unmistakable. ### Set Up Alerts for Yield Drops URL: https://www.tofupilot.com/guides/how-to-set-up-alerts-for-yield-drops-with-tofupilot Learn how TofuPilot alerts on yield drops automatically against each line's own baseline, and when to add a custom rule with your own numbers. A yield drop that goes unnoticed for a full shift can mean hundreds of scrapped units. TofuPilot watches first-pass yield on every procedure and notifies you when it falls out of that line's normal range, so you can intervene before the damage spreads. ## Why Real-Time Alerting Matters Production problems rarely announce themselves. A solder paste machine runs low and starts producing weak joints. A fixture contact wears down and adds resistance to every measurement. A new component lot has slightly different characteristics. In all these cases, the yield drop starts small and grows. The earlier you catch it, the fewer units are affected. Waiting for end-of-shift reports or weekly quality reviews is too slow. ## What TofuPilot Can Alert On TofuPilot monitors your production data and covers six conditions. | Alert type | What it catches | Automatic | |------------|-----------------|-----------| | Yield drop | First-pass yield falls out of the procedure's normal range | Yes | | Measurement drift | A measurement's mean shifts away from its own baseline | Yes | | Unit retest threshold | One unit is retested far more than the procedure's norm | Yes | | Golden sample failed | A known-good reference unit starts failing | Yes | | Failing sample passed | A known-bad reference unit starts passing | Yes | | Run failed | A streak of consecutive failed runs | Custom rule only | The two trend types catch broad degradation. The reference-sample types catch a test that has stopped discriminating at all, which is the failure mode that silently invalidates everything else. Retest thresholds catch the unit nobody wants to admit was retested nine times. ## Automatic Detection Sets No Threshold This is the part most people expect to configure and do not have to. There is no yield floor to enter. A floor that is right for a line running at 99% is wrong for a burn-in test that normally runs at 55%, and both are healthy. So TofuPilot measures what each procedure normally does — including how much that procedure normally *wobbles* — and alerts when the current smoothed yield departs from that baseline by more than the line's own variation. | Severity | Fires at | |----------|----------| | Info | ≥ 2σ | | Warning | ≥ 3σ | | Critical | ≥ 5σ | σ (sigma) here means "multiples of this line's own normal swing". A line that habitually moves between 50% and 60% has a wide normal swing, so wandering inside it does not alert. A line that sits rock-steady at 55% has a narrow one, and the same swing does alert. Two consequences are worth knowing. A procedure needs roughly **275 first-runs over the past 90 days** before automatic yield grading applies, so a brand-new line is silent until it has history. And on near-perfect lines, severity stays at info while fewer than three actual failures sit in recent memory, because above roughly 99.6% yield a single isolated failure is statistically dramatic and practically nothing. The full method is documented in [automatic detection](/docs/alerts/automatic-detection). ## When to Add a Custom Rule Automatic detection answers "is this unusual for this line?". A custom rule answers a different question: "is this below the number I promised someone?" Use one when the number comes from outside the data — a customer commitment, a ramp target, a line qualification gate. Custom yield rules are still expressed as a **drop in percentage points against the line's own recent baseline**, not as an absolute floor, and you pick which yield they grade: | Metric | What it counts | |--------|----------------| | First Pass Yield | Units whose first-ever run on the procedure passed. Retests cannot improve it. | | Last Pass Yield | Units whose last run passed. The state after rework. | | Run pass rate | Passing runs over all runs. Retest storms drag it down fast. | You set the drop that matters at each severity, and the scope — a whole procedure, one station, one part. See [alert rules](/docs/alerts/alert-rules) for the full configuration. ## Produce Clean, Structured Test Data Alerts are only as good as the data behind them. Write your OpenHTF tests with clear, measurable outputs so TofuPilot can track trends and trigger alerts on the right signals. ```python filename="test_charger_board.py" import openhtf as htf from openhtf.util import units from tofupilot.openhtf import TofuPilot @htf.measures( htf.Measurement("charge_voltage") .in_range(minimum=4.15, maximum=4.25) .with_units(units.VOLT), htf.Measurement("charge_current") .in_range(minimum=0.450, maximum=0.550) .with_units(units.AMPERE), htf.Measurement("thermal_shutdown_temp") .in_range(minimum=80, maximum=90) .with_units(units.DEGREE_CELSIUS), ) def test_charging_circuit(test): test.measurements.charge_voltage = 4.19 test.measurements.charge_current = 0.498 test.measurements.thermal_shutdown_temp = 85.2 def main(): test = htf.Test( test_charging_circuit, station_id="LINE-A-FCT-02", ) with TofuPilot(test): test.execute(test_start=lambda: "CHG-2026-03318") if __name__ == "__main__": main() ``` Each measurement with defined limits feeds TofuPilot's analytics. The tighter and more consistent your naming, the more useful your alerts will be — a measurement renamed halfway through the quarter starts a new series with no history, and a series with no history cannot be graded. ## Design Tests for Better Alerting You can make alerts more useful by how you structure your tests. ```python filename="test_led_driver.py" import openhtf as htf from openhtf.util import units from tofupilot.openhtf import TofuPilot @htf.measures( # Separate measurements for each channel make alerts specific htf.Measurement("led_current_ch1") .in_range(minimum=0.018, maximum=0.022) .with_units(units.AMPERE), htf.Measurement("led_current_ch2") .in_range(minimum=0.018, maximum=0.022) .with_units(units.AMPERE), htf.Measurement("led_current_ch3") .in_range(minimum=0.018, maximum=0.022) .with_units(units.AMPERE), htf.Measurement("led_current_ch4") .in_range(minimum=0.018, maximum=0.022) .with_units(units.AMPERE), ) def test_led_channels(test): test.measurements.led_current_ch1 = 0.0201 test.measurements.led_current_ch2 = 0.0198 test.measurements.led_current_ch3 = 0.0203 test.measurements.led_current_ch4 = 0.0200 def main(): test = htf.Test( test_led_channels, station_id="LINE-B-FCT-01", ) with TofuPilot(test): test.execute(test_start=lambda: "LED-2026-05519") if __name__ == "__main__": main() ``` Splitting measurements by channel (instead of a single pass/fail for all channels) means TofuPilot can alert you that channel 3 is drifting specifically. That's actionable. A generic "LED test failing more" is not. ## Respond to Alerts When an alert fires, act on it: 1. **Check the data** in TofuPilot. The alert carries the numbers that fired it and a chart of the metric; look at the trend and the distribution to understand what changed. 2. **Narrow down the scope.** Is it one station or all stations? One shift or all shifts? One part or the whole procedure? This points to the root cause faster than any single measurement will. 3. **Fix the root cause.** Recalibrate the equipment, replace the fixture, retrain the operator, quarantine the component lot. 4. **Let it close itself.** A yield alert resolves automatically once the metric recovers well inside the band. Resolving by hand instead snoozes that alert for an hour, which is what you want when you already know and are working on it — and not what you want as a habit. Alerts are a starting point, not the answer. They tell you something changed. The data in TofuPilot tells you what. ### Test Sequencing and Orchestration URL: https://www.tofupilot.com/guides/test-sequencing-and-orchestration-with-tofupilot Learn how to build repeatable hardware test sequences using TofuPilot with OpenHTF and Python for structured, automated test orchestration. # Test Sequencing and Orchestration with TofuPilot A hardware test isn't one check. It's a sequence: power up, wait for boot, measure voltages, run calibration, check communication, stress test, power down. TofuPilot captures each step with its measurements and timing, giving you a structured record of the full test flow. ## Why Test Sequencing Matters Running tests manually from a bench instrument works for prototypes. It doesn't work for production. Production testing needs: - **Repeatability**: Every unit goes through the exact same steps in the exact same order - **Speed**: No waiting for an operator to click "next" - **Data capture**: Every measurement recorded automatically with limits and pass/fail status - **Traceability**: A record of exactly what was tested, in what order, with what results Test orchestration means defining that sequence once and running it the same way every time. ## Test Sequence Architecture A well-structured test sequence follows a pattern: ``` ┌─────────────┐ │ Setup │ Power on, initialize instruments, identify DUT ├─────────────┤ │ Step 1 │ Measurement or action with pass/fail criteria ├─────────────┤ │ Step 2 │ Next measurement or action ├─────────────┤ │ ... │ Additional steps as needed ├─────────────┤ │ Teardown │ Power off, release instruments, upload results └─────────────┘ ``` Each step produces measurements. Each measurement has limits. The sequence stops on critical failures or continues through all steps depending on your strategy. ## Building a Test Sequence with OpenHTF OpenHTF is a Python framework designed for hardware test sequencing. TofuPilot integrates directly as an output callback. ```python filename="production_test_sequence.py" import openhtf as htf from openhtf.plugs import BasePlug from tofupilot.openhtf import TofuPilotClient class PowerSupplyPlug(BasePlug): """Controls the bench power supply.""" def setup(self): self.psu = connect_power_supply() def set_voltage(self, voltage): self.psu.write(f"VOLT {voltage}") def enable_output(self): self.psu.write("OUTP ON") def disable_output(self): self.psu.write("OUTP OFF") def teardown(self): self.disable_output() class DMMPlug(BasePlug): """Reads from the digital multimeter.""" def setup(self): self.dmm = connect_dmm() def measure_voltage(self): return float(self.dmm.query("MEAS:VOLT:DC?")) def measure_current(self): return float(self.dmm.query("MEAS:CURR:DC?")) # Step 1: Power rail verification @htf.measures( htf.Measurement("vcc_3v3").in_range(3.25, 3.35).with_units("V"), htf.Measurement("vcc_1v8").in_range(1.75, 1.85).with_units("V"), htf.Measurement("vcc_5v0").in_range(4.90, 5.10).with_units("V"), ) @htf.PhaseOptions(name="Power Rail Verification") def power_rail_check(test, psu: PowerSupplyPlug, dmm: DMMPlug): psu.set_voltage(12.0) psu.enable_output() time.sleep(0.5) # Wait for rails to stabilize test.measurements.vcc_3v3 = dmm.measure_voltage() # Switch DMM channel and measure other rails test.measurements.vcc_1v8 = dmm.measure_voltage() test.measurements.vcc_5v0 = dmm.measure_voltage() # Step 2: Current consumption @htf.measures( htf.Measurement("idle_current_ma").in_range(30, 60).with_units("mA"), htf.Measurement("active_current_ma").in_range(80, 150).with_units("mA"), ) @htf.PhaseOptions(name="Current Consumption") def current_check(test, psu: PowerSupplyPlug, dmm: DMMPlug): test.measurements.idle_current_ma = dmm.measure_current() * 1000 trigger_active_mode() time.sleep(0.2) test.measurements.active_current_ma = dmm.measure_current() * 1000 # Step 3: Communication check @htf.measures( htf.Measurement("uart_loopback").equals(True), htf.Measurement("spi_whoami").equals(0x68), ) @htf.PhaseOptions(name="Communication Interfaces") def comm_check(test): test.measurements.uart_loopback = verify_uart_loopback() test.measurements.spi_whoami = read_spi_register(0x75) def main(): test = htf.Test( power_rail_check, current_check, comm_check, ) test.add_output_callbacks(TofuPilotClient()) test.execute(lambda: input("Scan DUT serial: ")) ``` Every phase runs in order. Every measurement is captured. The full sequence uploads to TofuPilot when the test completes. ## Building a Test Sequence with the Python Client If you're not using OpenHTF, the TofuPilot Python client handles test sequencing directly. ```python filename="sequence_with_client.py" from tofupilot import TofuPilotClient import time client = TofuPilotClient() serial = input("Scan DUT serial: ") steps = [] # Step 1: Power rails psu.enable(12.0) time.sleep(0.5) vcc_3v3 = dmm.measure_voltage(channel=1) vcc_1v8 = dmm.measure_voltage(channel=2) steps.append({ "name": "Power Rail Verification", "step_type": "measurement", "status": 3.25 <= vcc_3v3 <= 3.35 and 1.75 <= vcc_1v8 <= 1.85, "measurements": [ {"name": "vcc_3v3", "value": vcc_3v3, "unit": "V", "limit_low": 3.25, "limit_high": 3.35}, {"name": "vcc_1v8", "value": vcc_1v8, "unit": "V", "limit_low": 1.75, "limit_high": 1.85}, ], }) # Step 2: Functional check boot_ok = wait_for_boot(timeout=5) steps.append({ "name": "Boot Sequence", "step_type": "measurement", "status": boot_ok, "measurements": [ {"name": "boot_success", "value": boot_ok, "unit": "bool"}, ], }) # Upload the complete sequence run_passed = all(s["status"] for s in steps) client.create_run( procedure_id="BOARD-FUNCTIONAL-V3", unit_under_test={"serial_number": serial}, run_passed=run_passed, steps=steps, ) ``` ## Sequence Design Best Practices | Practice | Why | |----------|-----| | Test cheap things first | If a power rail is shorted, don't waste time running communication checks | | Group related measurements | Keep all voltage checks in one step, all current checks in another | | Use consistent step names | "Power Rail Verification" across all procedures, not "Voltage Check" in one and "Rail Test" in another | | Include setup/teardown | Always power down the DUT at the end, even if the test fails | | Set limits on every measurement | A measurement without limits can't be trended or analyzed | ## Handling Sequence Failures Two strategies for what happens when a step fails: **Fail-fast**: Stop the sequence immediately on the first failure. Use this for safety-critical tests or when a failure in step 1 makes later steps meaningless (e.g., power rail failure means no point testing communication). **Run-all**: Continue through all steps even if one fails. Use this when you want complete diagnostic data (e.g., knowing which 3 out of 20 measurements failed helps root cause analysis). OpenHTF supports both via phase options. The TofuPilot Python client lets you implement either pattern in your test logic. ## What TofuPilot Stores for Each Sequence Every test run in TofuPilot captures: | Data | Purpose | |------|---------| | Procedure ID | Which test sequence was run | | Serial number | Which unit was tested | | Overall pass/fail | Did the sequence pass? | | Steps with measurements | Every step, every measurement, every limit | | Timestamps | When the test started and ended | | Station ID | Which station ran the test | | Duration | How long the sequence took | This structured data is what enables trending, comparison, and analytics across your entire production history. ### Data-Driven Manufacturing Testing URL: https://www.tofupilot.com/guides/data-driven-manufacturing-testing-with-tofupilot Learn how to replace opinion-based test decisions with data-driven manufacturing quality control using TofuPilot's analytics platform. # Data-Driven Manufacturing Testing with TofuPilot Most hardware test decisions are still made on gut feeling. "I think yield dropped last week." "This component lot seems worse." "The night shift probably isn't following the procedure." TofuPilot replaces opinions with measurements, so every quality decision is backed by data. ## The Problem with Opinion-Based Testing Hardware test operations generate enormous amounts of data. But in most organizations, that data goes into local files and never gets analyzed systematically. Decisions are made based on: - The last failure someone remembers - Anecdotal reports from operators - Monthly summary reports that are outdated by the time they're reviewed - Institutional knowledge that walks out the door when engineers leave Data-driven testing means every decision (change limits, adjust process, qualify a supplier, release a batch) is backed by actual measurement data. ## What Data-Driven Testing Looks Like ### Setting Test Limits **Opinion-based**: "The spec says 3.3V +/- 5%, so let's use 3.135V to 3.465V as test limits." **Data-driven**: Measure 1,000 units. The distribution is centered at 3.31V with a standard deviation of 0.015V. Set test limits at 3.25V to 3.37V (4-sigma). This gives a Cpk of 1.33, balancing quality with false failure rates. ```python filename="calculate_limits.py" import numpy as np # Production measurement data from TofuPilot values = [3.31, 3.30, 3.32, 3.29, 3.31, 3.30, 3.33, 3.31, 3.28, 3.32] # ... (1000 values in practice) mean = np.mean(values) std = np.std(values, ddof=1) # Set limits at mean +/- 4 sigma for Cpk ~ 1.33 limit_low = round(mean - 4 * std, 3) limit_high = round(mean + 4 * std, 3) print(f"Mean: {mean:.3f} V") print(f"Std: {std:.4f} V") print(f"Recommended limits: {limit_low} V to {limit_high} V") ``` ### Qualifying a New Supplier **Opinion-based**: "The samples looked fine. Let's approve the supplier." **Data-driven**: Run 50 units with the new supplier's components through your full test suite. Compare measurement distributions against the baseline from your current supplier. | Measurement | Current supplier (mean) | New supplier (mean) | Shift | Verdict | |------------|------------------------|--------------------| ------|---------| | vcc_3v3 | 3.310 V | 3.305 V | -0.005 V | OK | | current_idle | 42.1 mA | 48.3 mA | +6.2 mA | Investigate | | boot_time | 340 ms | 335 ms | -5 ms | OK | The current draw shift is within limits but significant. Investigate the root cause before full qualification. ### Deciding When to Service a Fixture **Opinion-based**: "It's been 6 months. Time for scheduled maintenance." **Data-driven**: Track station-specific measurement variance over time. Service the fixture when variance exceeds a threshold, not on a calendar. | Station | Variance (week 1) | Variance (week 8) | Variance (week 16) | Action | |---------|-------------------|-------------------|--------------------| -------| | STN-01 | 0.008 | 0.009 | 0.011 | OK | | STN-02 | 0.007 | 0.012 | 0.025 | Service now | | STN-03 | 0.009 | 0.010 | 0.010 | OK | Station 2's variance tripled. Service it now. Stations 1 and 3 are fine. Skip their scheduled maintenance. ## Building a Data-Driven Test Culture ### Step 1: Centralize Everything No data-driven decisions are possible when test data lives in 15 different places. TofuPilot centralizes all test results from all stations, all procedures, all sites. ```python filename="centralize.py" from tofupilot import TofuPilotClient client = TofuPilotClient() # Every station, every test, every site pushes to TofuPilot client.create_run( procedure_id="ICT-BOARD-V4", unit_under_test={"serial_number": "PCB-20251087"}, run_passed=True, steps=[...], ) ``` ### Step 2: Define Key Metrics Pick the metrics that matter most to your operation: | Metric | What it tells you | How often to review | |--------|------------------|-------------------| | FPY by procedure | Which test is your bottleneck | Daily | | Cpk by measurement | Process capability | Weekly | | Failure pareto | Top quality issues | Daily | | Station FPY comparison | Equipment health | Weekly | | Yield trend | Direction of quality | Daily | ### Step 3: Make Decisions from Dashboards Replace meeting-based decisions with dashboard-based decisions. Instead of a weekly quality review where someone presents slides, open TofuPilot's dashboard and look at the actual data. Questions the dashboard answers directly: - "Should we release this batch?" Check FPY and measurement distributions. - "Is Station 3 performing?" Compare its metrics to other stations. - "Did the process change improve yield?" Compare before and after. - "Is this supplier's quality acceptable?" Compare measurement distributions. ### Step 4: Automate Decisions Where Possible Some decisions can be fully automated. Use TofuPilot's API to build automated gates: - Auto-release batches with FPY above 98% - Auto-flag stations with yield below 95% - Auto-reject units with any critical measurement failure - Auto-escalate when a new failure mode appears in the top 3 ## The Payoff Data-driven testing doesn't require new test equipment or radical process changes. It requires centralizing the data you're already collecting and building the habit of making decisions from dashboards instead of opinions. The typical result: yield improves 2-5% in the first quarter, not because you changed anything fundamental, but because you started seeing problems sooner and fixing them faster. ### Automated Test Reporting with TofuPilot URL: https://www.tofupilot.com/guides/automated-test-reporting-with-tofupilot Learn how to generate automated hardware test reports from TofuPilot data, replacing manual Excel and PDF report creation. # Automated Test Reporting with TofuPilot Test reports are necessary. Building them manually is not. TofuPilot stores all your test data in a structured format, so reports can be generated automatically instead of assembled by hand in Excel every Friday. ## The Manual Reporting Problem A typical weekly quality report requires: 1. Export data from each test station (15 min per station) 2. Merge exports into one spreadsheet (30 min) 3. Calculate FPY, failure pareto, and measurement stats (45 min) 4. Build charts (30 min) 5. Write summary and observations (30 min) 6. Format and distribute (15 min) That's 3+ hours every week. For a quality engineer who should be analyzing data, not formatting cells. ## What TofuPilot Automates TofuPilot replaces steps 1-4 entirely. The data is already centralized. The metrics are already computed. The charts are already drawn. | Manual step | TofuPilot equivalent | |------------|---------------------| | Export from each station | Data uploads automatically | | Merge into one spreadsheet | Already in one database | | Calculate FPY | Computed in real time | | Build charts | Built-in dashboards | | Failure pareto | Auto-generated from test results | | Measurement distributions | Histograms per measurement | ## Types of Test Reports ### Per-Unit Test Report A complete record of one unit's test results. Useful for: - Customer acceptance documentation - Warranty records - Regulatory submissions Contents: serial number, all test procedures run, all measurements with limits, pass/fail status, timestamps. ### Batch Summary Report Aggregate quality data for a production batch. Useful for: - Production release decisions - Supplier quality reviews - Management dashboards Contents: batch size, FPY, failure pareto, measurement distributions, Cpk values. ### Station Performance Report How each test station is performing. Useful for: - Maintenance planning - Capacity planning - Station qualification Contents: FPY per station, cycle time, failure modes, measurement distributions. ## Generating Reports via the API Use TofuPilot's API to pull data for custom report formats. ```python filename="batch_report.py" from tofupilot import TofuPilotClient from datetime import datetime, timedelta client = TofuPilotClient() # Pull all runs for a procedure in the last 7 days runs = client.get_runs( procedure_id="FINAL-FUNCTIONAL-V3", limit=500, ) # Calculate batch statistics total = len(runs) passed = sum(1 for r in runs if r["run_passed"]) fpy = passed / total if total > 0 else 0 # Failure mode breakdown failure_modes = {} for run in runs: if not run["run_passed"]: for step in run.get("steps", []): if not step["status"]: mode = step["name"] failure_modes[mode] = failure_modes.get(mode, 0) + 1 print(f"Batch Report - FINAL-FUNCTIONAL-V3") print(f"Period: Last 7 days") print(f"Total units: {total}") print(f"Passed: {passed}") print(f"FPY: {fpy:.1%}") print() print("Top failure modes:") for mode, count in sorted(failure_modes.items(), key=lambda x: -x[1]): print(f" {mode}: {count} ({count/total:.1%})") ``` ### Generating a CSV Export ```python filename="export_csv.py" import csv from tofupilot import TofuPilotClient client = TofuPilotClient() runs = client.get_runs( procedure_id="FINAL-FUNCTIONAL-V3", limit=1000, ) with open("test_report.csv", "w", newline="") as f: writer = csv.writer(f) writer.writerow(["Serial", "Date", "Status", "Step", "Measurement", "Value", "Unit", "Low Limit", "High Limit", "Pass"]) for run in runs: serial = run["unit_under_test"]["serial_number"] date = run["created_at"] status = "PASS" if run["run_passed"] else "FAIL" for step in run.get("steps", []): for m in step.get("measurements", []): writer.writerow([ serial, date, status, step["name"], m["name"], m["value"], m.get("unit", ""), m.get("limit_low", ""), m.get("limit_high", ""), "PASS" if step["status"] else "FAIL", ]) print(f"Exported {len(runs)} runs to test_report.csv") ``` ## Report Scheduling ### Option A: Cron-Based Reports ```python filename="weekly_report.py" # Run every Monday at 7 AM via cron # 0 7 * * 1 python3 /opt/reports/weekly_report.py from tofupilot import TofuPilotClient import smtplib from email.mime.text import MIMEText client = TofuPilotClient() # Generate report content runs = client.get_runs(procedure_id="FINAL-FUNCTIONAL-V3", limit=500) total = len(runs) passed = sum(1 for r in runs if r["run_passed"]) fpy = passed / total if total > 0 else 0 report = f"""Weekly Test Report - FINAL-FUNCTIONAL-V3 Units tested: {total} First-pass yield: {fpy:.1%} """ # Send via email msg = MIMEText(report) msg["Subject"] = f"Weekly Test Report - FPY {fpy:.1%}" msg["From"] = "reports@yourcompany.com" msg["To"] = "quality-team@yourcompany.com" with smtplib.SMTP("smtp.yourcompany.com") as server: server.send_message(msg) ``` ### Option B: Dashboard Links Instead of generating PDF reports, share TofuPilot dashboard links. The recipient sees live data, not a snapshot from when the report was generated. Advantages over static reports: - Always up to date - Interactive (filter, drill down, compare) - No generation step needed - No distribution step needed ## Replacing the Weekly Quality Meeting Instead of spending the first 30 minutes of your quality meeting presenting data, open TofuPilot's dashboard and start the conversation from the data. The meeting shifts from "here's what happened" to "here's what we should do about it." ### Manage Subcontractor Test Quality URL: https://www.tofupilot.com/guides/how-to-manage-subcontractor-test-quality-with-tofupilot Give subcontractors your test procedures, collect their results in your TofuPilot workspace, and maintain full visibility over outsourced production quality. You design the product. Your contract manufacturer builds it. But when quality problems slip through, your brand takes the hit. TofuPilot lets you share test procedures with subcontractors and collect every result in your workspace, so you have the same visibility as if you were on the factory floor. ## The OEM/CM Visibility Problem Most OEMs get a weekly yield report from their CM, maybe a spreadsheet, maybe an email with a number. That's not enough. You need measurement-level data for every unit: what was tested, what passed, what failed, and what the actual values were. Without this data, you can't tell whether a 96% yield means the process is healthy or whether it's hiding a measurement that's drifting toward the limit. By the time the weekly report shows a yield drop, hundreds of bad units may have shipped. ## Set Up the Workflow The setup is straightforward. You write the test procedures, the subcontractor runs them, and results upload to your TofuPilot workspace. 1. **You** write and version-control the OpenHTF test scripts 2. **You** provide the subcontractor with the scripts and a TofuPilot API key scoped to your workspace 3. **The subcontractor** installs the scripts on their test stations and runs production 4. **TofuPilot** collects every run with station metadata identifying the subcontractor's site The subcontractor doesn't need access to your TofuPilot dashboard. They just run the tests. You see everything. ## Write Tests the Subcontractor Will Run Keep test scripts self-contained. The subcontractor's operators shouldn't need to modify anything. Use `station_id` to identify the CM's stations in your data. ```python filename="test_motor_driver.py" import openhtf as htf from openhtf.util import units from tofupilot.openhtf import TofuPilot @htf.measures( htf.Measurement("vbus_voltage") .in_range(minimum=23.5, maximum=24.5) .with_units(units.VOLT), htf.Measurement("phase_current") .in_range(minimum=0.9, maximum=1.1) .with_units(units.AMPERE), htf.Measurement("pwm_frequency") .in_range(minimum=19500, maximum=20500) .with_units(units.HERTZ), ) def test_motor_output(test): test.measurements.vbus_voltage = 24.01 test.measurements.phase_current = 1.02 test.measurements.pwm_frequency = 20010 def main(): test = htf.Test( test_motor_output, station_id="CM-ACME-SZ-FCT-01", ) with TofuPilot(test): test.execute(test_start=lambda: "MDR-2026-04417") if __name__ == "__main__": main() ``` Prefix the station ID with the subcontractor's name or code (like `CM-ACME-SZ`) so you can filter by CM in TofuPilot. ## Provide Deployment Instructions Package everything the subcontractor needs: | Item | Purpose | |------|---------| | Test scripts (Git repo or zip) | The exact procedures to run | | `requirements.txt` | Python dependencies including `openhtf` and `tofupilot` | | Environment setup guide | How to set the `TOFUPILOT_API_KEY` environment variable | | Station naming convention | How to set `station_id` for each machine | | Instrument configuration | VISA addresses, serial ports, fixture pin maps | The subcontractor sets the API key as an environment variable. The TofuPilot client picks it up automatically. ```bash filename="setup_env.sh" # The subcontractor sets this once per station export TOFUPILOT_API_KEY=tp_live_xxxxxxxxxxxx # The TofuPilot client reads it automatically # No code changes needed ``` ## Handle Multiple Subcontractors If you work with more than one CM, use distinct station ID prefixes for each: ```python filename="test_sensor_board.py" import openhtf as htf from openhtf.util import units from tofupilot.openhtf import TofuPilot @htf.measures( htf.Measurement("temp_accuracy") .in_range(minimum=-0.5, maximum=0.5) .with_units(units.DEGREE_CELSIUS), htf.Measurement("humidity_accuracy_pct") .in_range(minimum=-3.0, maximum=3.0), ) def test_sensor_accuracy(test): test.measurements.temp_accuracy = 0.12 test.measurements.humidity_accuracy_pct = -1.3 def main(): test = htf.Test( test_sensor_accuracy, # CM-BETA identifies this subcontractor's facility station_id="CM-BETA-GDL-EOL-03", ) with TofuPilot(test): test.execute(test_start=lambda: "SNS-2026-09102") if __name__ == "__main__": main() ``` This gives you clean filtering: see all of CM-BETA's results, compare CM-ACME vs CM-BETA yield, or drill into a specific station. ## Monitor Subcontractor Quality in TofuPilot With all CMs uploading to your workspace, TofuPilot gives you: - **Yield by subcontractor** so you can compare CM performance at a glance - **Measurement distributions per CM** to catch process differences. If CM-ACME's voltage readings are shifted compared to CM-BETA, one of them has a calibration issue. - **Failure Pareto per site** to see whether different CMs struggle with different tests - **Real-time run feed** so you see results as they happen, not in a weekly report ## Keep Procedures in Sync When you update a test procedure, every CM needs the new version. Use Git tags or release versions so you can verify which version each site is running. TofuPilot tracks the procedure used for each run, so you can filter by procedure version and confirm the rollout. Don't let CMs modify test limits or skip tests. The procedures you ship are the single source of truth. If a CM reports that a test is failing too often, investigate the root cause instead of loosening limits. ### Production Test Validation at Scale URL: https://www.tofupilot.com/guides/production-test-validation-at-scale-with-tofupilot Learn how to validate your production test process at scale using TofuPilot's statistical analysis, Cpk tracking, and yield monitoring. # Production Test Validation at Scale with TofuPilot Running a test that works for 10 prototypes is different from running it for 10,000 production units. At scale, you need to validate not just the product, but the test process itself. Are your limits correct? Is your test repeatable? Are you catching real defects without creating false failures? TofuPilot's analytics help answer these questions. ## What Production Test Validation Means Production test validation (PVT) answers three questions: 1. **Are the test limits correct?** Limits that are too tight cause false failures. Limits that are too loose let defective units ship. 2. **Is the test repeatable?** The same unit tested twice should give the same result. 3. **Is the test effective?** Does it catch the defects it's supposed to catch? ## Step 1: Analyze Measurement Distributions After running your test on the first 100-200 production units, analyze the measurement distributions in TofuPilot. ```python filename="distribution_analysis.py" import numpy as np from tofupilot import TofuPilotClient client = TofuPilotClient() runs = client.get_runs( procedure_id="FINAL-FUNCTIONAL-V3", limit=200, ) # Extract measurement values vcc_values = [] for run in runs: for step in run.get("steps", []): for m in step.get("measurements", []): if m["name"] == "vcc_3v3": vcc_values.append(m["value"]) values = np.array(vcc_values) print(f"N: {len(values)}") print(f"Mean: {np.mean(values):.4f} V") print(f"Std: {np.std(values, ddof=1):.4f} V") print(f"Min: {np.min(values):.4f} V") print(f"Max: {np.max(values):.4f} V") print(f"Range: {np.max(values) - np.min(values):.4f} V") ``` What to look for: | Observation | Action | |------------|--------| | Distribution centered within limits, Cpk > 1.33 | Limits are well-set | | Distribution skewed toward one limit | Investigate process bias | | Distribution wider than expected | Tighten process or widen limits | | Outliers beyond 3-sigma | Investigate those specific units | | Bimodal distribution | Two populations, likely mixed lots | ## Step 2: Calculate Process Capability Cpk tells you how well your process fits within the test limits. TofuPilot provides the measurement data; you calculate the Cpk. ```python filename="cpk_validation.py" import numpy as np def calculate_cpk(values, lsl, usl): mean = np.mean(values) std = np.std(values, ddof=1) cpu = (usl - mean) / (3 * std) cpl = (mean - lsl) / (3 * std) cpk = min(cpu, cpl) return cpk, mean, std # From TofuPilot data vcc_values = [3.30, 3.31, 3.29, 3.32, 3.30, 3.31, 3.29, 3.30, 3.33, 3.31] lsl, usl = 3.25, 3.35 cpk, mean, std = calculate_cpk(vcc_values, lsl, usl) print(f"Cpk: {cpk:.2f}") print(f"Mean: {mean:.3f} V") print(f"Std: {std:.4f} V") if cpk >= 1.67: print("Excellent process capability") elif cpk >= 1.33: print("Acceptable process capability") elif cpk >= 1.0: print("Marginal. Consider tightening process or widening limits") else: print("Poor capability. Action required") ``` | Cpk | Meaning | DPMO (approx) | |-----|---------|---------------| | 2.0 | Excellent | 0.002 | | 1.67 | Very good | 0.6 | | 1.33 | Good | 63 | | 1.0 | Marginal | 2,700 | | 0.67 | Poor | 45,500 | ## Step 3: Validate Test Repeatability (Gauge R&R) Test the same unit multiple times to measure your test system's repeatability. ```python filename="repeatability_test.py" from tofupilot import TofuPilotClient client = TofuPilotClient() # Test the same unit 30 times serial = "GRR-GOLDEN-001" for i in range(30): vcc = measure_voltage() client.create_run( procedure_id="GRR-FUNCTIONAL-V3", unit_under_test={"serial_number": serial}, run_passed=True, steps=[{ "name": "Power Rail", "step_type": "measurement", "status": True, "measurements": [{ "name": "vcc_3v3", "value": vcc, "unit": "V", "limit_low": 3.25, "limit_high": 3.35, }], }], ) ``` After 30 runs, analyze the spread in TofuPilot: | Metric | Target | Meaning | |--------|--------|---------| | GR&R % of tolerance | < 10% | Excellent measurement system | | GR&R % of tolerance | 10-30% | Acceptable, monitor | | GR&R % of tolerance | > 30% | Measurement system needs improvement | If your test measurement varies by 0.04V on the same unit and your tolerance is 0.10V, that's 40% GR&R. Your test is too noisy to reliably distinguish good from bad units. ## Step 4: Optimize Test Limits Use production data to optimize limits. The goal: catch real defects without rejecting good units. ### Tightening Limits If Cpk > 2.0 and you're seeing no false failures, your limits might be too loose. Tighter limits catch marginal units before they become field failures. ### Widening Limits If Cpk < 1.0 and you're seeing false failures (units that fail test but work fine in the field), your limits are too tight for your current process capability. ### Dynamic Limits Some teams use TofuPilot data to set limits based on the production distribution: ```python filename="dynamic_limits.py" # Calculate limits from production data mean = 3.310 std = 0.015 # 4-sigma limits for Cpk = 1.33 dynamic_low = mean - 4 * std # 3.250 dynamic_high = mean + 4 * std # 3.370 ``` ## Step 5: Monitor at Scale Once your test is validated, monitor it continuously. Scale introduces new variables: - Different operators - Different component lots across months - Fixture wear over thousands of cycles - Environmental changes (season, humidity) - Equipment calibration drift TofuPilot's trend dashboards surface these changes. Set up monitoring for: 1. **FPY trend**: Catch yield drops within hours 2. **Cpk trend**: Catch process capability degradation within days 3. **Measurement mean shift**: Catch drift before it causes failures 4. **Failure pareto changes**: Catch new failure modes early ## Validation Checklist Before approving a test for production volume: - [ ] Measurement distributions are normal (or expected shape) - [ ] Cpk > 1.33 for all critical measurements - [ ] GR&R < 30% for all measurements - [ ] No false failures in the last 200 units - [ ] Test catches known defect modes (verified with known-bad units) - [ ] Test cycle time meets throughput requirements - [ ] All stations produce equivalent results (cross-station correlation) ### What Is Generative Test Design URL: https://www.tofupilot.com/guides/what-is-generative-test-design Generative test design uses AI to create test plans, scripts, and measurement strategies from product specifications. Learn where it works today and where. # What Is Generative Test Design Generative test design uses AI to create test plans, test scripts, and measurement strategies from product specifications. Instead of an engineer manually translating a datasheet into test code, an AI system reads the requirements and generates the test structure. This guide covers what generative test design can do today, what it can't, and where it's heading. ## From Spec to Test Traditional test development follows this path: | Step | Who Does It | Time | |------|-----------|------| | Read product spec | Test engineer | Hours | | Identify testable requirements | Test engineer | Hours | | Choose measurements and limits | Test engineer | Hours | | Write test code | Test engineer | Days | | Debug and validate | Test engineer | Days | | Document test procedure | Test engineer | Hours | Generative test design compresses steps 1-4: | Step | Who Does It | Time | |------|-----------|------| | Feed spec to AI | Test engineer | Minutes | | AI generates test structure | AI | Seconds | | Engineer reviews and refines | Test engineer | Hours | | Validate on hardware | Test engineer | Hours | | Document (auto-generated from code) | AI | Minutes | The engineer's role shifts from writing to reviewing. The AI handles the translation from requirements to code. The engineer validates that the translation is correct and the tests work on real hardware. ## What Works Today | Capability | Status | Quality | |-----------|--------|---------| | Generate OpenHTF phases from a text description | Works | Good for standard patterns (voltage check, communication test) | | Suggest measurement limits from a datasheet | Works | Needs engineer review for margin decisions | | Generate test scripts from similar existing tests | Works well | AI can adapt a power supply test to a different model | | Create test procedure documentation from code | Works | Accurate but may need formatting adjustments | | Generate fixture designs from board geometry | Early research | Not production-ready | ## What Doesn't Work Yet | Capability | Why Not | |-----------|---------| | Generate tests for novel products with no precedent | AI needs examples to learn from | | Choose between test methods (ICT vs FCT vs boundary scan) | Requires physical understanding of the product | | Design test fixtures | Requires 3D geometry and mechanical knowledge | | Set limits that account for manufacturing variation | Requires production data that doesn't exist yet for new products | | Replace domain expertise for safety-critical tests | Regulatory requirements demand human judgment | ## Example: Spec to Test Code Given this specification fragment: > Output voltage: 12V +/- 5% > Output current: 0-2A > Ripple: < 50mV peak-to-peak > Efficiency: > 85% at full load > Operating temperature: -20C to 60C An AI generates: ```python filename="generated_test.py" import openhtf as htf from openhtf.util import units @htf.measures( htf.Measurement("output_voltage_V") .in_range(minimum=11.4, maximum=12.6) .with_units(units.VOLT), htf.Measurement("ripple_mVpp") .in_range(maximum=50) .with_units(units.MILLIVOLT), htf.Measurement("efficiency_percent") .in_range(minimum=85) .with_units(units.PERCENT), ) def phase_output_validation(test): """Validate output characteristics per product specification.""" test.measurements.output_voltage_V = 12.03 test.measurements.ripple_mVpp = 28.4 test.measurements.efficiency_percent = 91.2 ``` The AI correctly computed 12V +/- 5% as 11.4-12.6V, chose appropriate units, and structured the code following OpenHTF patterns. The engineer still needs to: - Add instrument control code (the AI generated placeholder values) - Review the limits (should there be marginal bands?) - Add phases for load testing at 0A and 2A - Add temperature-dependent tests if required ## Prerequisites - Python 3.10+ - OpenHTF installed (`pip install openhtf`) - TofuPilot Python SDK installed (`pip install tofupilot`) ## Step 1: Start with a Template Generative test design works best when the AI has examples to learn from. Start with a working test and ask the AI to adapt it for a new product. ```python filename="template_test.py" import openhtf as htf from openhtf.util import units @htf.measures( htf.Measurement("output_voltage_V") .in_range(minimum=4.75, maximum=5.25) .with_units(units.VOLT), ) def phase_voltage_check(test): """Template: measure output voltage against spec.""" test.measurements.output_voltage_V = 5.01 ``` From this template, the AI can generate variants for different voltage levels, add current measurements, include ripple checks, and create multi-phase test sequences. ## Step 2: Review and Refine AI-generated test code needs human review before deployment. Check for: | Check | What to Verify | |-------|---------------| | Limits | Do they match the spec exactly? Are margins appropriate? | | Units | Are units correct and consistent? | | Measurement names | Are they descriptive and consistent with your naming convention? | | Phase structure | Are phases logically grouped? Is the sequence correct? | | Missing tests | Did the AI miss any requirement from the spec? | | Instrument control | Is the SCPI command sequence correct for your instruments? | ## Step 3: Log and Learn Connect the generated test to TofuPilot. As production data accumulates, it feeds back into better test generation for the next product. ```python filename="template_test.py" from tofupilot.openhtf import TofuPilot test = htf.Test(phase_voltage_check) with TofuPilot(test): test.execute(test_start=lambda: input("Scan serial: ")) ``` ## Where This Is Heading | Timeframe | Capability | |-----------|-----------| | Now | Generate test code from text descriptions and adapt existing templates | | Near-term | Generate test plans from product specs with requirement traceability | | Mid-term | AI suggests which tests to add based on failure patterns in similar products | | Long-term | End-to-end: spec in, validated test system out, including fixture design and instrument selection | Generative test design won't eliminate the need for test engineers. It will make them faster. The engineers who learn to use AI as a design tool will build test systems in days instead of weeks. ### What Is a Test Copilot URL: https://www.tofupilot.com/guides/what-is-a-test-copilot A test copilot is an AI assistant that helps engineers write tests, analyze failures, and optimize limits. Learn what it does and where the technology is. # What Is a Test Copilot A test copilot is an AI assistant purpose-built for test engineering. Like GitHub Copilot for software development, a test copilot helps engineers write test scripts, analyze failure data, suggest measurement limits, and debug test sequences. This guide covers what a test copilot does, how it differs from general-purpose AI, and where the technology is heading. ## What a Test Copilot Does | Capability | What It Looks Like | |-----------|-------------------| | Test script generation | Describe what you want to test in plain English, get working OpenHTF phases | | Limit suggestion | Analyze production data and recommend measurement limits with margins | | Failure analysis | Ask "why are units failing phase_voltage_check?" and get root cause analysis | | Code review | Flag common test script mistakes (wrong validators, missing units, bad plug patterns) | | Documentation | Generate test procedure documents from code | | Troubleshooting | Describe a symptom, get diagnostic steps based on test history | ## Why Test Engineering Needs a Specialized Copilot General-purpose AI (ChatGPT, Claude, GitHub Copilot) can write Python code, but test engineering has domain-specific patterns that generic models get wrong: | Pattern | General AI Gets It Wrong | Test Copilot Gets It Right | |---------|------------------------|--------------------------| | OpenHTF plug injection | Uses type hints (doesn't work) | Uses `@htf.plug()` decorator | | Measurement validators | Invents `.at_least()` (doesn't exist) | Uses `.in_range(minimum=x)` | | Test limits | Picks round numbers | Derives from datasheet specs or production data | | Instrument control | Generic SCPI examples | Instrument-specific command sequences | | Failure analysis | Generic debugging advice | Correlates with test data patterns | A test copilot is trained on (or has access to) test frameworks, instrument documentation, measurement science, and production data. It speaks the language of FPY, Cpk, DUT, and SCPI. ## Current State of the Technology ### What Exists Today | Product | What It Does | Scope | |---------|-------------|-------| | NI Nigel AI | AI assistant trained on NI hardware, software, and test methodologies | NI ecosystem only | | Flux Copilot | AI assistant for PCB design (not test) | Hardware design, not test | | GitHub Copilot | Code completion for any language | Generic, not test-aware | | TofuPilot + Claude/ChatGPT | AI assistants with access to TofuPilot data via MCP | Open, framework-agnostic | ### What's Emerging | Capability | Status | |-----------|--------| | AI-generated test plans from product specifications | Research phase | | Automatic limit optimization from production data | Early products in semiconductor | | Natural language test specification | Academic (ASE 2025 conference papers) | | AI-driven root cause analysis from test data | Deployed in automotive (Acerta, QualityLine) | | Agentic test execution (AI decides what to test next) | Concept phase, Forrester defined category Q3 2025 | ## How a Test Copilot Fits Into the Workflow | Workflow Stage | Without Copilot | With Copilot | |---------------|----------------|-------------| | Writing test script | Engineer writes from scratch or copies from template | Describe the test, copilot generates phases with measurements and limits | | Setting limits | Engineer reads datasheet, picks values | Copilot analyzes production data, suggests limits with 3-sigma margins | | Debugging failures | Engineer reviews logs, guesses root cause | Copilot correlates failure patterns across thousands of runs | | Reviewing test coverage | Engineer manually checks requirements vs test steps | Copilot flags requirements not covered by any test phase | | Optimizing cycle time | Engineer profiles phases manually | Copilot identifies redundant tests and suggests skip conditions | ## Prerequisites - Python 3.10+ - OpenHTF installed (`pip install openhtf`) - TofuPilot Python SDK installed (`pip install tofupilot`) ## Example: From Description to Test Code Today, an engineer can describe a test to an AI assistant and get working code. The quality depends on the assistant's knowledge of the test framework. A well-trained test copilot turns this description: > "Test a 5V power supply. Check output voltage is 4.9-5.1V, ripple is under 50mV, and efficiency is above 90%." Into this code: ```python filename="power_supply_test.py" import openhtf as htf from openhtf.util import units @htf.measures( htf.Measurement("output_voltage_V") .in_range(minimum=4.9, maximum=5.1) .with_units(units.VOLT), htf.Measurement("ripple_mV") .in_range(maximum=50) .with_units(units.MILLIVOLT), htf.Measurement("efficiency_percent") .in_range(minimum=90) .with_units(units.PERCENT), ) def phase_power_supply_validation(test): """Validate power supply output characteristics.""" test.measurements.output_voltage_V = 5.02 test.measurements.ripple_mV = 28.3 test.measurements.efficiency_percent = 93.1 ``` ```python filename="power_supply_test.py" from tofupilot.openhtf import TofuPilot test = htf.Test(phase_power_supply_validation) with TofuPilot(test): test.execute(test_start=lambda: input("Scan serial: ")) ``` The copilot knows to use `.in_range()` (not `.at_least()`), to include `units`, and to structure the test with TofuPilot integration. ## Where This Is Heading | Timeframe | Capability | |-----------|-----------| | Now | AI generates test scripts from descriptions, reviews code for common mistakes | | Near-term | AI suggests limits based on production data, identifies root causes from failure patterns | | Mid-term | AI optimizes test sequences (adaptive testing), generates test plans from product specs | | Long-term | Autonomous test systems that design, execute, and optimize tests with minimal human input | The test copilot won't replace test engineers. It will handle the repetitive parts (writing boilerplate, analyzing large datasets, setting initial limits) so engineers can focus on test strategy, fixture design, and solving the hard problems that require physical intuition. ### How to Manage NPI Testing with TofuPilot URL: https://www.tofupilot.com/guides/how-to-manage-npi-testing-with-tofupilot Structure your test procedures across EVT, DVT, and PVT phases. Refine measurement limits using early data and track procedure versions as your product matures. New product introduction means your test procedures start loose and get tighter as the design matures. TofuPilot tracks every version of your tests across EVT, DVT, and PVT, so you can see how limits evolved and use real production data to set final specifications. ## Testing Needs at Each NPI Phase Each phase has different goals, and your tests should reflect that. | Phase | Goal | Test approach | |-------|------|--------------| | EVT | Validate the design works | Wide limits, characterization measurements, data collection | | DVT | Prove reliability and compliance | Tightened limits, environmental stress tests, margin analysis | | PVT | Confirm manufacturing readiness | Production-ready limits, yield optimization, process capability | During EVT, you're learning what the product actually does. During PVT, you're proving it can be built at volume. The test procedures need to evolve accordingly. ## EVT: Characterize with Wide Limits In EVT, you don't know the true distribution of your measurements yet. Set limits wide enough to collect data without rejecting units unnecessarily. The goal is characterization, not screening. ```python filename="test_amplifier_evt.py" import openhtf as htf from openhtf.util import units from tofupilot.openhtf import TofuPilot @htf.measures( htf.Measurement("gain") .in_range(minimum=18.0, maximum=26.0) .doc("EVT: wide limits for characterization"), htf.Measurement("noise_floor") .in_range(maximum=-60.0) .doc("EVT: upper bound only, collecting distribution data"), htf.Measurement("input_impedance") .in_range(minimum=40.0, maximum=60.0) .with_units(units.OHM) .doc("EVT: centered on 50 ohm nominal, wide tolerance"), ) def test_amplifier_performance(test): test.measurements.gain = 21.4 test.measurements.noise_floor = -72.3 test.measurements.input_impedance = 49.1 def main(): test = htf.Test( test_amplifier_performance, station_id="LAB-BENCH-01", ) with TofuPilot(test): test.execute(test_start=lambda: "AMP-EVT-0012") if __name__ == "__main__": main() ``` Upload every EVT run to TofuPilot. Even with only 20-50 units, the measurement histograms show the natural distribution of your design. This data is what you'll use to set DVT limits. ## DVT: Tighten Limits Based on EVT Data After EVT, you have real measurement distributions in TofuPilot. Use them to set informed limits. Look at the histogram for each measurement: the mean tells you where the design centers, the spread tells you how much to allow. ```python filename="test_amplifier_dvt.py" import openhtf as htf from openhtf.util import units from tofupilot.openhtf import TofuPilot @htf.measures( htf.Measurement("gain") .in_range(minimum=20.0, maximum=23.0) .doc("DVT: tightened from EVT data, Cpk target > 1.33"), htf.Measurement("noise_floor") .in_range(maximum=-65.0) .doc("DVT: tightened based on EVT distribution"), htf.Measurement("input_impedance") .in_range(minimum=45.0, maximum=55.0) .with_units(units.OHM) .doc("DVT: narrowed from 40-60 to 45-55 based on EVT Cpk"), ) def test_amplifier_performance(test): test.measurements.gain = 21.2 test.measurements.noise_floor = -73.1 test.measurements.input_impedance = 50.3 def main(): test = htf.Test( test_amplifier_performance, station_id="LAB-BENCH-02", ) with TofuPilot(test): test.execute(test_start=lambda: "AMP-DVT-0087") if __name__ == "__main__": main() ``` The gain limit narrowed from 18-26 (EVT) to 20-23 (DVT) because EVT data showed the design naturally centers around 21 with low spread. TofuPilot's Cpk and measurement statistics make this analysis straightforward. ## PVT: Lock Production-Ready Limits PVT is the final gate before mass production. Limits should be tight enough to catch defective units but not so tight that they kill yield on good units. Use DVT data to set limits that achieve your Cpk target. ```python filename="test_amplifier_pvt.py" import openhtf as htf from openhtf.util import units from tofupilot.openhtf import TofuPilot @htf.measures( htf.Measurement("gain") .in_range(minimum=20.2, maximum=22.8) .doc("PVT: production limits, Cpk > 1.67"), htf.Measurement("noise_floor") .in_range(maximum=-67.0) .doc("PVT: production limit with margin"), htf.Measurement("input_impedance") .in_range(minimum=46.0, maximum=54.0) .with_units(units.OHM) .doc("PVT: final production limits"), htf.Measurement("thd_pct") .in_range(maximum=0.5) .doc("PVT: added THD test for production screening"), ) def test_amplifier_performance(test): test.measurements.gain = 21.3 test.measurements.noise_floor = -71.8 test.measurements.input_impedance = 50.1 test.measurements.thd_pct = 0.12 def main(): test = htf.Test( test_amplifier_performance, station_id="SZ-L1-FCT-01", ) with TofuPilot(test): test.execute(test_start=lambda: "AMP-PVT-0341") if __name__ == "__main__": main() ``` Notice the PVT version also adds a new measurement (`thd_pct`) that wasn't in EVT or DVT. As you learn more about the product through testing, you add tests that catch failure modes discovered during validation. ## Version-Control Your Procedure Evolution Keep every NPI phase in version control. Tag releases so you can trace which limits were active when any unit was tested. ``` v0.1.0 EVT - initial characterization limits v0.2.0 EVT - added noise floor measurement v1.0.0 DVT - tightened all limits from EVT data v1.1.0 DVT - added environmental stress sequence v2.0.0 PVT - production-ready limits, added THD v2.0.1 PVT - adjusted impedance limit after line trial ``` TofuPilot tracks which procedure version produced each run. You can filter by version to compare yield between EVT and DVT limits, or to see the impact of a specific limit change. ## Use TofuPilot Data to Drive Limit Decisions The key advantage of uploading every run from every phase is that TofuPilot builds a complete picture of your product's behavior over time. - **EVT histograms** show the design's natural distribution before any limit optimization - **DVT Cpk values** tell you whether your tightened limits have enough margin for volume production - **PVT yield trends** confirm the limits work on real production lines, not just lab benches - **Cross-phase comparison** shows how the measurement distribution changed as the design and process matured Don't set limits in a spreadsheet and hope for the best. Use the actual measurement data from each NPI phase to make informed decisions. TofuPilot keeps all of it in one place, from your first EVT prototype through PVT line trials and into mass production. ## Transition to Mass Production When PVT is complete and limits are locked, the same test procedure and TofuPilot integration carry straight into production. There's no handoff gap. The production team inherits: - Validated test procedures with data-driven limits - Historical measurement distributions for comparison - Established station configurations and naming conventions - Alert thresholds tuned during PVT line trials The NPI data stays in TofuPilot alongside production data, giving you a continuous record from first prototype to millionth unit. ### What Is a Continuous Test Stack URL: https://www.tofupilot.com/guides/what-is-a-continuous-test-stack A continuous test stack connects test development, execution, data collection, and analytics into one integrated workflow. Learn what it includes and how. # What Is a Continuous Test Stack A continuous test stack connects every stage of manufacturing test into one integrated workflow: writing tests, deploying them to stations, executing on the production floor, collecting data, and analyzing results. Instead of disconnected tools at each stage, a continuous stack feeds data from production back into test development, creating a loop that improves test quality over time. ## The Fragmented Stack Problem Most manufacturing test operations look like this: | Stage | Tool | Data Format | Connection to Next Stage | |-------|------|------------|------------------------| | Write tests | Text editor or IDE | Python/LabVIEW files | Manual copy to station | | Deploy to stations | USB drive or shared folder | Executable or script | None | | Execute tests | Test executive (OpenHTF, TestStand) | Internal format | CSV export or local DB | | Store results | File server or spreadsheet | CSV, Excel | Manual report generation | | Analyze | Excel, custom scripts | Charts, pivot tables | Email to engineering | | Improve | Engineer reads report, updates test | Back to step 1 | Manual, delayed | Every handoff between stages is manual. Data changes format. Context is lost. The feedback loop from production data back to test improvement takes weeks or months. ## The Continuous Stack A continuous test stack eliminates the handoffs: | Stage | How It Works | Feedback Loop | |-------|-------------|--------------| | Write tests | Python scripts in Git | Production failure data informs what to test next | | Deploy to stations | pip install or Docker pull | Version-controlled, reproducible | | Execute tests | OpenHTF runs phases, streams results | Real-time operator UI | | Store results | Automatic upload to TofuPilot | Structured, searchable, traceable | | Analyze | Live dashboards, automated alerts | Immediate visibility | | Improve | Engineer sees data, updates test, pushes to Git | Hours, not weeks | The key difference: data flows automatically from execution to analysis, and insights flow back to development. There's no manual export step, no Excel processing, no emailed reports. ## Components of a Continuous Test Stack | Component | Purpose | Open Source Option | |-----------|---------|-------------------| | Test framework | Sequence and execute test phases | OpenHTF, pytest | | Version control | Track test script changes | Git | | Package management | Deploy test scripts to stations | pip, Docker | | Data platform | Store and query test results | TofuPilot | | Operator interface | Production floor UI | TofuPilot streaming | | Analytics | Yield, SPC, Pareto, trends | TofuPilot Analytics | | Alerting | Notify on yield drops or drift | TofuPilot alerts | ## Prerequisites - Python 3.10+ - OpenHTF installed (`pip install openhtf`) - TofuPilot Python SDK installed (`pip install tofupilot`) - Git for version control ## Step 1: Write Tests in Git Keep test scripts in a Git repository. Every change is tracked, reviewable, and reversible. ```python filename="tests/production_test.py" import openhtf as htf from openhtf.util import units @htf.measures( htf.Measurement("output_voltage_V") .in_range(minimum=4.9, maximum=5.1) .with_units(units.VOLT), htf.Measurement("current_draw_mA") .in_range(minimum=90, maximum=110) .with_units(units.MILLIAMPERE), ) def phase_electrical(test): """Production electrical test.""" test.measurements.output_voltage_V = 5.01 test.measurements.current_draw_mA = 99.3 ``` ## Step 2: Deploy with pip Package your tests as a Python package. Stations install or update with one command. ```bash filename="terminal" pip install --upgrade your-test-package ``` This replaces the USB drive workflow. Every station runs the same version. Updates are traceable. ## Step 3: Stream Results to TofuPilot Connect the test to TofuPilot. Results flow automatically from every station to a central platform. ```python filename="tests/production_test.py" from tofupilot.openhtf import TofuPilot test = htf.Test(phase_electrical) with TofuPilot(test): test.execute(test_start=lambda: input("Scan serial: ")) ``` ## Step 4: Close the Loop The continuous part: use production data to improve tests. | Signal in TofuPilot | Action | |---------------------|--------| | Phase with 0% failure rate for 10K units | Consider removing from production test | | Measurement consistently at 99% of limit | Investigate design margin, adjust limit | | Yield drop after test script update | Git blame the change, revert if needed | | New failure mode appears | Add a test phase to catch it | | Station 3 has lower yield than stations 1-2 | Investigate fixture or equipment on station 3 | Each action results in a code change in Git, a new deployment to stations, and the cycle continues. The stack is continuous because data flows in a loop, not a line. ## Continuous Stack vs Point Solutions | Approach | Pros | Cons | |----------|------|------| | Point solutions (TestStand + custom DB + Excel) | Each tool is best-in-class for its stage | No integration, manual handoffs, data silos | | Continuous stack (OpenHTF + Git + TofuPilot) | Integrated data flow, fast feedback loops | Requires upfront architecture decisions | | Custom-built everything | Perfectly tailored | Expensive to build and maintain | The continuous stack approach trades some per-tool flexibility for end-to-end integration. The ROI comes from faster feedback loops: when you can go from "yield dropped" to "root cause identified" to "fix deployed" in hours instead of weeks, the investment pays for itself quickly. ### Offline Test Data Sync with TofuPilot URL: https://www.tofupilot.com/guides/offline-test-data-sync-with-tofupilot Learn how to collect hardware test data in offline or air-gapped environments and sync it to TofuPilot when connectivity is restored. # Offline Test Data Sync with TofuPilot Not every test station has an internet connection. Cleanrooms, field deployments, secure facilities, and factory floors with spotty WiFi all need to capture test data reliably. TofuPilot's Python client handles offline scenarios by letting you store results locally and sync when connectivity returns. ## When Offline Testing Happens | Scenario | Why it's offline | |----------|-----------------| | Air-gapped facility | Security requirements prohibit internet access | | Field testing | Remote location with no connectivity | | Cleanroom | No network access inside the controlled environment | | Factory floor | Unreliable WiFi, can't depend on it for every test | | Mobile test station | Traveling between sites | In all these cases, the test must run and record data regardless of network status. You can't tell a production line to stop because the WiFi is down. ## Architecture for Offline-First Testing ``` ┌──────────────┐ ┌──────────────┐ ┌──────────────┐ │ Test Station │────▶│ Local Store │────▶│ TofuPilot │ │ (runs tests) │ │ (JSON files) │ │ (cloud) │ └──────────────┘ └──────────────┘ └──────────────┘ Offline When connected ``` The test station writes results to local storage. A sync process uploads them to TofuPilot when a connection is available. ## Step 1: Store Results Locally Write test results to JSON files on the local machine. Each file represents one test run. ```python filename="offline_test.py" import json import os from datetime import datetime OFFLINE_DIR = "/data/test-results/pending" os.makedirs(OFFLINE_DIR, exist_ok=True) def run_test(serial_number): """Run the test and store results locally.""" # Run your actual test measurements vcc = measure_voltage(channel=1) current = measure_current() result = { "procedure_id": "BOARD-FUNCTIONAL-V3", "unit_under_test": {"serial_number": serial_number}, "run_passed": 3.25 <= vcc <= 3.35 and 30 <= current <= 60, "tested_at": datetime.utcnow().isoformat(), "steps": [{ "name": "Power Rail Check", "step_type": "measurement", "status": 3.25 <= vcc <= 3.35, "measurements": [{ "name": "vcc_3v3", "value": vcc, "unit": "V", "limit_low": 3.25, "limit_high": 3.35, }], }, { "name": "Current Draw", "step_type": "measurement", "status": 30 <= current <= 60, "measurements": [{ "name": "idle_current_ma", "value": current, "unit": "mA", "limit_low": 30, "limit_high": 60, }], }], } filename = f"{serial_number}_{datetime.utcnow().strftime('%Y%m%d_%H%M%S')}.json" filepath = os.path.join(OFFLINE_DIR, filename) with open(filepath, "w") as f: json.dump(result, f, indent=2) print(f"Result saved locally: {filepath}") return result # Run tests regardless of network status run_test("UNIT-5501") run_test("UNIT-5502") ``` ## Step 2: Sync When Connected A separate sync script uploads pending results to TofuPilot and moves them to a "synced" directory. ```python filename="sync_results.py" import json import os import shutil from tofupilot import TofuPilotClient PENDING_DIR = "/data/test-results/pending" SYNCED_DIR = "/data/test-results/synced" os.makedirs(SYNCED_DIR, exist_ok=True) def sync_pending_results(): """Upload all pending results to TofuPilot.""" client = TofuPilotClient() pending_files = sorted(os.listdir(PENDING_DIR)) if not pending_files: print("No pending results to sync.") return print(f"Syncing {len(pending_files)} pending results...") for filename in pending_files: filepath = os.path.join(PENDING_DIR, filename) with open(filepath) as f: result = json.load(f) try: client.create_run( procedure_id=result["procedure_id"], unit_under_test=result["unit_under_test"], run_passed=result["run_passed"], steps=result["steps"], ) # Move to synced directory shutil.move(filepath, os.path.join(SYNCED_DIR, filename)) print(f"Synced: {filename}") except Exception as e: print(f"Failed to sync {filename}: {e}") # Leave in pending for retry remaining = len(os.listdir(PENDING_DIR)) print(f"Sync complete. {remaining} files still pending.") sync_pending_results() ``` ## Step 3: Automate the Sync Run the sync script automatically when connectivity is available. ### Option A: Cron Job ```bash filename="crontab" # Try to sync every 15 minutes */15 * * * * /usr/bin/python3 /opt/test-station/sync_results.py >> /var/log/test-sync.log 2>&1 ``` ### Option B: Network Event Trigger ```python filename="network_watcher.py" import subprocess import time def has_internet(): try: subprocess.check_call( ["curl", "-s", "--max-time", "5", "https://app.tofupilot.com/api/health"], stdout=subprocess.DEVNULL, ) return True except subprocess.CalledProcessError: return False while True: if has_internet(): subprocess.run(["python3", "sync_results.py"]) time.sleep(300) # Check every 5 minutes ``` ## Data Integrity Offline sync introduces a risk: what if a file gets corrupted or a sync partially fails? | Risk | Mitigation | |------|-----------| | File corruption | Write to temp file first, then rename (atomic write) | | Duplicate upload | TofuPilot deduplicates by procedure + serial + timestamp | | Partial sync failure | Files stay in pending until successfully uploaded | | Clock drift on offline machines | Use monotonic timestamps for ordering, UTC for absolute time | ## When to Use This Pattern Use offline-first testing when: - Your test station can't guarantee network connectivity - Network latency would slow down test cycle time - You need tests to run even during network outages - Security requirements prevent direct cloud access from the test floor For stations with reliable connectivity, push results directly to TofuPilot from within the test script. The offline pattern adds complexity that's only justified when connectivity is unreliable. ### What Is Autonomous Test Closure URL: https://www.tofupilot.com/guides/what-is-autonomous-test-closure Autonomous test closure uses AI to determine when a unit has been tested enough. Learn how it works, where it applies, and what data it needs. # What Is Autonomous Test Closure Autonomous test closure is the concept that an AI system can determine when sufficient testing has been performed on a unit, and stop the test sequence early without human intervention. Instead of running every unit through every test step, the system evaluates test results in real time and decides: this unit has been tested enough, we're confident it's good. This guide covers how the concept works, where it applies, and what data it needs. ## The Problem: Over-Testing Most manufacturing test sequences are static. Every unit runs through every test phase regardless of how clearly it's passing. The result: good units are tested longer than necessary. | Scenario | Test Time | Units Per Hour | |----------|----------|----------------| | Full test (all phases) | 60 seconds | 60 | | Early closure on high-confidence units | 35 seconds (average) | 103 | | Improvement | 42% faster | 72% more throughput | In semiconductor testing, this concept (called "adaptive test" or "predictive binning") has been deployed for over a decade, reducing test time by 10-50%. In discrete manufacturing, it's an emerging concept. ## How It Works Autonomous test closure evaluates two things after each test phase: 1. **Confidence level**: Based on measurements so far, how confident are we this unit is good? 2. **Remaining risk**: What's the probability that a remaining test phase would catch a defect the previous phases missed? | After Phase | Measurements So Far | Confidence | Decision | |-------------|-------------------|-----------|----------| | Phase 1: Power-up | Current within spec, voltage nominal | 70% | Continue | | Phase 2: Communication | All buses responding correctly | 85% | Continue | | Phase 3: Analog check | All channels within 2% of nominal | 97% | Close early (skip phases 4-5) | | Phase 4: Stress test | (Skipped) | - | - | | Phase 5: Final validation | (Skipped) | - | - | The confidence threshold is configurable. Higher thresholds mean fewer tests are skipped. Lower thresholds mean faster throughput but higher risk. ## Prerequisites | Requirement | Why | |-------------|-----| | Historical test data (10,000+ units) | Train the confidence model | | Low correlation between skipped and kept tests | If phases are redundant, skipping is safe | | Stable, mature product | New products need full test data | | No regulatory requirement for 100% testing | Some industries can't skip tests | ## Where It Applies | Applicable | Not Applicable | |-----------|---------------| | High-volume consumer electronics | Medical devices (FDA requires defined test protocol) | | Mature products with stable yield | New product introduction (need all data) | | Tests with redundant coverage | Safety-critical tests (hipot, leakage) | | Products with high FPY (>97%) | Products with unstable processes | ## Levels of Autonomous Closure | Level | How It Decides | Risk | |-------|---------------|------| | 1. Rule-based | "If phases 1-3 pass, skip phase 5" | Low (engineer defines rules) | | 2. Statistical | "If all measurements are within 2-sigma, skip remaining" | Medium (data-driven threshold) | | 3. ML-based | Trained model predicts pass probability from partial test data | Medium-high (model accuracy dependent) | | 4. Fully autonomous | System continuously learns and adjusts skip decisions | High (requires robust safeguards) | ## Safeguards Autonomous test closure increases throughput but introduces risk. Safeguards are essential: | Safeguard | Purpose | |-----------|---------| | Audit sampling | Run full test on 5-10% of early-closed units | | Escape monitoring | Track field returns for early-closed vs full-tested units | | Confidence floor | Never close below a minimum confidence threshold | | Phase lock-in | Some phases (safety, regulatory) can never be skipped | | Automatic reversion | If audit failures increase, revert to full testing | ## The Data Foundation Autonomous test closure depends on structured test data. Every measurement, every limit, every pass/fail result from every unit builds the dataset the confidence model learns from. ```python filename="test_with_data.py" import openhtf as htf from openhtf.util import units @htf.measures( htf.Measurement("supply_voltage_V") .in_range(minimum=4.9, maximum=5.1) .with_units(units.VOLT), htf.Measurement("current_mA") .in_range(minimum=90, maximum=110) .with_units(units.MILLIAMPERE), ) def phase_power(test): """Phase 1: Power-up check.""" test.measurements.supply_voltage_V = 5.01 test.measurements.current_mA = 99.5 @htf.measures( htf.Measurement("comm_status").equals("PASS"), ) def phase_communication(test): """Phase 2: Communication check.""" test.measurements.comm_status = "PASS" @htf.measures( htf.Measurement("analog_ch1_V") .in_range(minimum=2.4, maximum=2.6) .with_units(units.VOLT), ) def phase_analog(test): """Phase 3: Analog measurement. High-confidence units may close here.""" test.measurements.analog_ch1_V = 2.50 ``` ```python filename="test_with_data.py" from tofupilot.openhtf import TofuPilot test = htf.Test( phase_power, phase_communication, phase_analog, ) with TofuPilot(test): test.execute(test_start=lambda: input("Scan serial: ")) ``` TofuPilot stores every measurement from every run. This structured dataset is what autonomous closure models need: thousands of units with complete measurement profiles and known pass/fail outcomes. ## The Path Forward Autonomous test closure is at the intersection of adaptive testing, predictive quality, and AI. The technology exists in semiconductor testing. Applying it to discrete manufacturing requires: 1. **Structured test data** (measurements with units, limits, serial traceability) 2. **Historical depth** (thousands of units with complete test profiles) 3. **Correlation analysis** (which early phases predict overall pass/fail) 4. **Confidence modeling** (statistical or ML-based pass prediction) 5. **Safeguard infrastructure** (audit sampling, escape monitoring, automatic reversion) Start by collecting the data. The intelligence comes later. ### Video and Log Sync for Hardware Tests URL: https://www.tofupilot.com/guides/video-and-log-synchronization-for-hardware-tests Learn how to attach video recordings, log files, and supplementary data to hardware test runs in TofuPilot for synchronized review. # Video and Log Synchronization for Hardware Tests Measurements tell you what happened. Videos and logs tell you why. A failed vibration test makes more sense when you can watch the unit rattle loose. A firmware crash is easier to debug when the serial console log is right next to the test result. TofuPilot stores attachments alongside measurements. ## Why Attachments Matter A measurement value of "FAIL" tells you something broke. But it doesn't tell you: - What the unit physically looked like during the test - What the firmware logged before the crash - What the oscilloscope waveform looked like - What the operator saw on their screen Attachments bridge this gap. They turn a test result from a number into a complete record of what happened. ## Types of Test Attachments | Attachment type | Use case | Format | |----------------|----------|--------| | Video recording | Visual inspection, mechanical tests | MP4, AVI | | Serial console log | Firmware debug, boot sequence | TXT, LOG | | Oscilloscope capture | Waveform analysis, timing | PNG, CSV | | Thermal image | Hot spot detection | JPEG, PNG | | Test station log | Test software debug | TXT, LOG | | Configuration file | DUT or fixture settings | JSON, YAML | | Photo | Physical defect documentation | JPEG, PNG | ## Attaching Files to Test Runs ### With the Python Client ```python filename="test_with_attachments.py" from tofupilot import TofuPilotClient client = TofuPilotClient() client.create_run( procedure_id="VIBRATION-SCREENING", unit_under_test={"serial_number": "UNIT-7832"}, run_passed=False, steps=[{ "name": "Random Vibration 20-2000Hz", "step_type": "measurement", "status": False, "measurements": [{ "name": "resonance_freq_hz", "value": 847, "unit": "Hz", "limit_low": 900, "limit_high": 1500, }], }], attachments=[ "recordings/unit-7832-vibration.mp4", "logs/unit-7832-accel.csv", "captures/unit-7832-spectrum.png", ], ) ``` ### Capturing Video Automatically Record video during the test and attach it to the run. ```python filename="video_capture.py" import subprocess import os def start_recording(output_path): """Start recording from a USB camera.""" proc = subprocess.Popen([ "ffmpeg", "-f", "v4l2", "-i", "/dev/video0", "-t", "120", # max 2 minutes "-y", output_path, ], stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL) return proc def stop_recording(proc): """Stop the recording.""" proc.terminate() proc.wait() # Usage in a test video_path = f"/tmp/test_{serial}.mp4" recorder = start_recording(video_path) # Run the test... run_test() stop_recording(recorder) # Attach to TofuPilot run client.create_run( procedure_id="MECHANICAL-TEST", unit_under_test={"serial_number": serial}, run_passed=passed, steps=steps, attachments=[video_path], ) ``` ### Capturing Serial Console Logs ```python filename="serial_logger.py" import serial import threading class SerialLogger: def __init__(self, port, baudrate=115200): self.ser = serial.Serial(port, baudrate, timeout=1) self.log = [] self.running = False def start(self): self.running = True self.log = [] self.thread = threading.Thread(target=self._read_loop) self.thread.start() def stop(self): self.running = False self.thread.join() self.ser.close() def _read_loop(self): while self.running: line = self.ser.readline().decode("utf-8", errors="replace") if line: self.log.append(line) def save(self, path): with open(path, "w") as f: f.writelines(self.log) # Usage logger = SerialLogger("/dev/ttyUSB0") logger.start() # Run the test... run_test() logger.stop() log_path = f"/tmp/console_{serial}.log" logger.save(log_path) # Attach to TofuPilot client.create_run( procedure_id="FIRMWARE-VALIDATION", unit_under_test={"serial_number": serial}, run_passed=passed, steps=steps, attachments=[log_path], ) ``` ## Synchronized Review When you open a test run in TofuPilot, all attachments are available alongside the measurements. This synchronized view lets you: 1. See that `resonance_freq_hz` failed at 847 Hz (below the 900 Hz limit) 2. Watch the video to see the enclosure flexing during the test 3. Open the accelerometer CSV to see the raw vibration data 4. Check the spectrum plot to identify the resonant mode All in one place, for one run, with one click. ## When to Attach What Not every test needs video and logs. Attachments add storage and complexity. Use them where they add diagnostic value. | Test type | Recommended attachments | |-----------|------------------------| | Vibration/shock | Video, accelerometer data | | Firmware validation | Serial console log | | Visual inspection | Photo of DUT | | Power testing | Oscilloscope captures | | Environmental testing | Temperature/humidity logs | | Burn-in | All logs (long-duration, hard to reproduce) | For routine production tests (ICT, basic functional), measurements alone are usually sufficient. Save attachments for tests where visual or log context helps debugging. ## Storage Considerations | Practice | Why | |----------|-----| | Compress videos before attaching | Reduce storage costs | | Keep attachments under 50MB per run | Practical upload/download speed | | Use PNG for plots, JPEG for photos | PNG for clarity, JPEG for size | | Only attach to failing runs (optionally) | Save storage on passing runs where attachments are rarely reviewed | | Include timestamps in log files | Correlate log events with test step timing | ### Track Battery Cycling Test Data URL: https://www.tofupilot.com/guides/how-to-track-battery-cycling-test-data-with-tofupilot Learn how to log charge/discharge cycling data, track capacity fade, and monitor cell performance across production using TofuPilot. # How to Track Battery Cycling Test Data with TofuPilot Battery cycling tests generate thousands of data points per cell: voltage curves, current profiles, temperature readings, capacity measurements across hundreds of cycles. TofuPilot stores all of it in a structured, queryable format so you can track capacity fade, compare cells, and catch quality issues before they reach the pack. ## What Battery Cycling Data Looks Like A single cell cycling test produces: | Data point | Per cycle | Over 500 cycles | |-----------|-----------|-----------------| | Voltage vs. time | ~1000 samples | 500,000 samples | | Current vs. time | ~1000 samples | 500,000 samples | | Temperature | ~100 samples | 50,000 samples | | Charge capacity (Ah) | 1 value | 500 values | | Discharge capacity (Ah) | 1 value | 500 values | | Coulombic efficiency | 1 value | 500 values | Multiply that by hundreds or thousands of cells in production, and you're looking at a data management problem that spreadsheets can't handle. ## Logging Cycling Data to TofuPilot ### Per-Cycle Upload Upload results after each charge/discharge cycle completes. This gives you real-time visibility into cell performance. ```python filename="battery_cycle_test.py" from tofupilot import TofuPilotClient client = TofuPilotClient() def log_cycle(cell_serial, cycle_number, charge_ah, discharge_ah, temp_max, voltage_curve): coulombic_eff = discharge_ah / charge_ah * 100 if charge_ah > 0 else 0 capacity_retention = discharge_ah / nominal_capacity * 100 client.create_run( procedure_id=f"CELL-CYCLING-C{cycle_number}", unit_under_test={"serial_number": cell_serial}, run_passed=capacity_retention > 80 and temp_max < 45, steps=[{ "name": f"Cycle {cycle_number}", "step_type": "measurement", "status": capacity_retention > 80, "measurements": [ {"name": "charge_capacity_ah", "value": charge_ah, "unit": "Ah", "limit_low": 2.8}, {"name": "discharge_capacity_ah", "value": discharge_ah, "unit": "Ah", "limit_low": 2.8}, {"name": "coulombic_efficiency_pct", "value": coulombic_eff, "unit": "%", "limit_low": 99.0}, {"name": "capacity_retention_pct", "value": capacity_retention, "unit": "%", "limit_low": 80.0}, {"name": "max_temperature_c", "value": temp_max, "unit": "°C", "limit_high": 45.0}, {"name": "voltage_curve_v", "value": voltage_curve, "unit": "V"}, ], }], ) ``` ### End-of-Life Summary Upload After cycling completes, upload a summary with key aging metrics. ```python filename="battery_eol_summary.py" client.create_run( procedure_id="CELL-CYCLING-SUMMARY", unit_under_test={"serial_number": cell_serial}, run_passed=final_capacity_retention > 80, steps=[{ "name": "Cycling Summary", "step_type": "measurement", "status": final_capacity_retention > 80, "measurements": [ {"name": "total_cycles", "value": 500, "unit": "cycles"}, {"name": "initial_capacity_ah", "value": 3.2, "unit": "Ah"}, {"name": "final_capacity_ah", "value": 2.72, "unit": "Ah"}, {"name": "capacity_retention_pct", "value": 85.0, "unit": "%", "limit_low": 80.0}, {"name": "avg_coulombic_efficiency", "value": 99.7, "unit": "%", "limit_low": 99.0}, {"name": "max_temperature_observed", "value": 42.3, "unit": "°C", "limit_high": 45.0}, ], }], ) ``` ## Tracking Capacity Fade Across Production The real value of centralized cycling data is comparing cells across production batches. TofuPilot's measurement trending shows: - **Capacity retention distribution**: Are all cells aging at the same rate? - **Batch-to-batch variation**: Does the new electrolyte formulation change the fade curve? - **Outlier detection**: Which cells are degrading faster than expected? If batch 47 cells show 90% retention at 300 cycles while batch 46 showed 94%, something changed in the manufacturing process. The data in TofuPilot tells you immediately. ## Cell Grading from Cycling Data Not all cells are created equal. Cycling data helps grade cells into bins for different applications. | Grade | Criteria | Application | |-------|----------|-------------| | A | Capacity > 3.1 Ah, retention > 95% at 200 cycles | EV packs | | B | Capacity > 2.9 Ah, retention > 90% at 200 cycles | Energy storage | | C | Below Grade B | Second-life applications | TofuPilot's measurement filters let you query cells by any combination of cycling metrics to assign grades automatically. ## Integration with Battery Cyclers Most battery cyclers (Arbin, Maccor, Neware, BioLogic) export data in CSV or proprietary formats. Parse the cycler output and upload to TofuPilot. ```python filename="arbin_import.py" import csv from tofupilot import TofuPilotClient client = TofuPilotClient() def import_arbin_data(csv_path, cell_serial): with open(csv_path) as f: reader = csv.DictReader(f) for row in reader: if row["Step_Type"] == "Discharge": client.create_run( procedure_id="CELL-CYCLING", unit_under_test={"serial_number": cell_serial}, run_passed=float(row["Discharge_Capacity(Ah)"]) > 2.8, steps=[{ "name": f"Cycle {row['Cycle_Index']}", "step_type": "measurement", "status": True, "measurements": [ {"name": "discharge_capacity_ah", "value": float(row["Discharge_Capacity(Ah)"]), "unit": "Ah"}, {"name": "charge_capacity_ah", "value": float(row["Charge_Capacity(Ah)"]), "unit": "Ah"}, {"name": "max_voltage_v", "value": float(row["Voltage(V)"]), "unit": "V"}, ], }], ) ``` ## Safety Monitoring Battery testing has unique safety requirements. TofuPilot helps track safety-critical measurements: - **Maximum temperature**: Cells approaching thermal runaway thresholds - **Voltage anomalies**: Cells that don't reach full charge voltage or drop too fast - **Capacity jumps**: Sudden capacity changes that indicate internal shorts - **Impedance growth**: Rising internal resistance indicating degradation Set tight limits on these safety measurements. A cell that passes capacity specs but shows abnormal temperature behavior needs investigation before it goes into a pack. ### What Is Predictive Quality in Mfg Test URL: https://www.tofupilot.com/guides/what-is-predictive-quality-in-manufacturing-test Predictive quality uses production data to catch defects before they happen. Learn how it works, what data it needs, and how test results feed prediction. # What Is Predictive Quality in Manufacturing Test Predictive quality uses historical test data and process data to forecast defects before they occur. Instead of catching bad units at end-of-line testing, predictive quality identifies the conditions that produce bad units and flags them earlier in the process. This guide covers how predictive quality works, what data it needs, and how structured test results feed prediction models. ## Reactive vs Predictive Quality | Approach | When Defects Are Found | Cost | |----------|----------------------|------| | Reactive (inspect and reject) | After the unit is built | High (scrap, rework, field returns) | | Statistical (SPC) | When a process drifts out of control | Medium (catches trends, not individual units) | | Predictive (ML-based) | Before the defect occurs | Low (prevents defective units from being built) | Traditional quality control catches defects after they happen. SPC catches trends before they produce defects. Predictive quality goes further: it uses patterns in upstream data to predict which specific units or batches will fail downstream tests. ## How Predictive Quality Works The core idea: upstream test data contains signals that correlate with downstream failures. | Step | What Happens | |------|-------------| | 1. Collect data | Every measurement, at every test stage, for every unit | | 2. Find correlations | ML models identify which upstream measurements predict downstream failures | | 3. Build prediction model | Train a model on historical pass/fail data with upstream features | | 4. Deploy inline | Run predictions on live test data as units move through production | | 5. Act on predictions | Flag high-risk units for additional inspection or route to rework | ### Example A power supply fails output ripple testing at end-of-line. Analysis of historical data shows that units with input capacitor ESR above 45 milliohms at incoming inspection are 8x more likely to fail ripple testing. A predictive model flags these units at IQC before they're assembled. ## What Data Feeds Predictive Quality | Data Source | What It Provides | Stage | |------------|-----------------|-------| | Incoming quality (IQC) | Component measurements, supplier lot data | Before assembly | | In-process quality (IPQC) | SPI paste volume, AOI defect counts, reflow profile | During assembly | | Functional test (FCT) | Electrical measurements with limits | After assembly | | End-of-line test (EOL) | Final pass/fail, measurement values | Before shipping | | Environmental data | Temperature, humidity on the production floor | Continuous | | Equipment data | Fixture cycle count, instrument calibration age | Continuous | The more structured your test data, the better the predictions. Measurements with units, limits, and serial number traceability are the foundation. ## Predictive Quality Use Cases | Use Case | Input Data | Prediction | Benefit | |----------|-----------|-----------|---------| | Skip testing | Upstream measurements | Unit will pass downstream test | 10-50% test time reduction | | Early warning | Process trends | Batch will have high failure rate | Catch before full batch is built | | Supplier quality | IQC measurements by vendor | Lot will cause downstream failures | Reject lots at incoming | | Limit optimization | Measurement distributions | Current limits are too wide or too tight | Reduce false failures and escapes | | Field failure prediction | Production test data | Unit will fail within warranty period | Tighten limits or add screening | ## Prerequisites - Python 3.10+ - OpenHTF installed (`pip install openhtf`) - TofuPilot Python SDK installed (`pip install tofupilot`) ## Step 1: Capture Structured Test Data Predictive quality starts with clean, consistent test data. Every measurement needs a name, value, unit, and limits. ```python filename="production_test.py" import openhtf as htf from openhtf.util import units @htf.measures( htf.Measurement("input_capacitor_esr_mOhm") .in_range(maximum=50) .with_units(units.OHM), htf.Measurement("output_ripple_mV") .in_range(maximum=30) .with_units(units.MILLIVOLT), htf.Measurement("efficiency_percent") .in_range(minimum=90) .with_units(units.PERCENT), ) def phase_electrical_test(test): """Capture measurements that feed predictive models.""" test.measurements.input_capacitor_esr_mOhm = 38.2 test.measurements.output_ripple_mV = 22.1 test.measurements.efficiency_percent = 93.4 ``` ## Step 2: Log Everything to TofuPilot Every run uploads to TofuPilot with serial number, measurements, limits, and pass/fail status. This structured data is what predictive models need. ```python filename="production_test.py" from tofupilot.openhtf import TofuPilot test = htf.Test(phase_electrical_test) with TofuPilot(test): test.execute(test_start=lambda: input("Scan serial: ")) ``` ## Step 3: Use TofuPilot Data for Analysis TofuPilot tracks measurement distributions, correlations, and trends across all test stages. Open the Analytics tab to identify: - **Measurement correlations** between upstream and downstream tests - **Failure patterns** by supplier lot, station, or time period - **Distribution shifts** that precede failure rate increases - **Marginal results** that predict future failures This data is the starting point for building predictive models. The structured format (measurements with units, limits, and serial traceability) eliminates the data cleaning step that typically consumes 80% of ML project time. ## Predictive Quality vs Traditional Quality | Aspect | Traditional | Predictive | |--------|------------|-----------| | When defects are found | After they happen | Before they happen | | Test strategy | Test everything the same way | Adapt testing based on risk | | Data use | Compliance reporting | Real-time decision making | | Limit setting | From datasheet or engineering judgment | From production data and ML models | | ROI measurement | Defect rate reduction | Cost avoidance (prevented scrap, skipped tests) | ## Getting Started Predictive quality doesn't require a massive ML infrastructure on day one. The progression is: | Level | What You Do | What You Need | |-------|------------|--------------| | 1. Collect | Log all measurements with units and limits | OpenHTF + TofuPilot | | 2. Visualize | Review distributions, correlations, and trends | TofuPilot Analytics | | 3. Correlate | Identify upstream predictors of downstream failures | Export data, run correlation analysis | | 4. Predict | Build and deploy ML models on live data | Data science team + production data | | 5. Automate | Act on predictions inline (skip tests, flag units) | Model deployment infrastructure | Most teams get significant value from levels 1-3 alone. Just visualizing the correlation between incoming component measurements and end-of-line failures reveals actionable insights without building any ML models. ### What Is End-of-Line Testing URL: https://www.tofupilot.com/guides/what-is-end-of-line-testing-with-tofupilot End-of-line testing validates every unit before it ships. Learn what EOL testing covers, how to structure it in Python, and how to track results with TofuPilot. # What Is End-of-Line Testing with TofuPilot End-of-line (EOL) testing is the final functional check a unit goes through before it leaves the production line. It catches defects that earlier process steps missed, and it produces the test record that proves the unit works. This guide covers what EOL testing involves, how to build an EOL test in Python, and how to log results to TofuPilot automatically. ## What End-of-Line Testing Covers EOL testing validates that a finished product meets its functional specification. It runs after assembly is complete and before packaging. A typical EOL test checks: | Check | Example | |-------|---------| | Power-up | Unit draws expected current at nominal voltage | | Communication | Serial, I2C, or CAN bus responds correctly | | Sensor output | Readings fall within calibrated limits | | Actuator response | Motor spins, valve opens, relay clicks | | Firmware version | Matches the released build | | Safety | Leakage current below regulatory threshold | EOL testing is pass/fail. Every unit gets the same sequence. The goal is coverage and speed, not deep characterization. ## EOL vs Other Test Stages | Stage | When | Purpose | |-------|------|---------| | Incoming inspection | Before assembly | Verify raw materials and components | | In-circuit test (ICT) | After SMT | Check solder joints and component placement | | Functional test (FCT) | After assembly | Validate subsystem behavior | | End-of-line test | After final assembly | Confirm the complete product works | | Outgoing quality audit | After EOL | Sample-based verification for shipping | EOL testing is the last gate. If a unit fails here, it goes to rework or scrap. If it passes, it ships. ## Prerequisites - Python 3.10+ - OpenHTF installed (`pip install openhtf`) - TofuPilot Python SDK installed (`pip install tofupilot`) ## Step 1: Define the Test Phases Each EOL check becomes an OpenHTF phase. Keep phases short and independent so failures are easy to isolate. ```python filename="eol_test.py" import openhtf as htf from openhtf.util import units @htf.measures( htf.Measurement("supply_current_mA") .in_range(minimum=45, maximum=55) .with_units(units.MILLIAMPERE) ) def phase_power_up(test): """Verify the unit draws expected current at 5V.""" test.measurements.supply_current_mA = 50.2 @htf.measures( htf.Measurement("firmware_version").equals("2.4.1") ) def phase_firmware_check(test): """Confirm firmware matches the released build.""" test.measurements.firmware_version = "2.4.1" @htf.measures( htf.Measurement("sensor_reading") .in_range(minimum=2.8, maximum=3.3) .with_units(units.VOLT) ) def phase_sensor_output(test): """Check that the onboard sensor reads within spec.""" test.measurements.sensor_reading = 3.05 ``` ## Step 2: Assemble and Run the Test Wire the phases into a test and connect TofuPilot. Every run uploads automatically. ```python filename="eol_test.py" from tofupilot.openhtf import TofuPilot test = htf.Test( phase_power_up, phase_firmware_check, phase_sensor_output, ) with TofuPilot(test): test.execute(test_start=lambda: input("Scan serial number: ")) ``` When the operator scans a serial number, OpenHTF runs all three phases in order. TofuPilot logs the result, measurements, and pass/fail status. ## Step 3: Track Results in TofuPilot TofuPilot tracks EOL test results automatically. Open the Analytics tab to see: - **First pass yield** across shifts and stations - **Measurement distributions** with limit overlays - **Failure Pareto** showing which phases fail most often - **Unit traceability** linking each serial number to its test record This data feeds directly into quality reviews and customer audits without manual reporting. ## When to Use EOL Testing EOL testing makes sense when: - Every unit must be individually verified before shipment - Regulatory or customer requirements demand test records per serial number - The product has multiple subsystems that interact after final assembly - Field returns need to be traceable back to production test data For low-volume, high-mix production, EOL tests often run on a single station with an operator. For high-volume lines, EOL stations run unattended with automated fixturing. ### Hipot and Dielectric Withstand Testing URL: https://www.tofupilot.com/guides/hipot-and-dielectric-withstand-testing-with-tofupilot Learn how to automate hipot and dielectric withstand safety tests, log results to TofuPilot, and maintain compliance records. # Hipot and Dielectric Withstand Testing with TofuPilot Hipot (high-potential) testing verifies that a product's electrical insulation can withstand voltage stress without breakdown. It's required by UL, IEC, and most safety standards for any product connected to mains power. TofuPilot logs every hipot result for production traceability and compliance. ## What Hipot Testing Verifies Hipot testing applies a high voltage between isolated circuits (typically line-to-ground or primary-to-secondary) and measures the leakage current. If current stays below the threshold and no breakdown occurs, the insulation is adequate. | Parameter | Typical value | Standard | |-----------|--------------|----------| | Test voltage (AC) | 1000-3000 Vrms | IEC 60950, IEC 62368 | | Test voltage (DC) | 1414-4243 Vdc | IEC 60601 (medical) | | Duration | 1-60 seconds | Varies by standard | | Leakage current limit | 0.5-10 mA | Varies by product class | | Ramp time | 0.5-3 seconds | Prevents voltage spikes | ## Automating Hipot Tests ### Connecting to a Hipot Tester Most programmable hipot testers (Chroma, Associated Research, GW Instek) support SCPI commands over GPIB, RS-232, or LAN. ```python filename="hipot_test.py" import pyvisa from tofupilot import TofuPilotClient rm = pyvisa.ResourceManager() hipot = rm.open_resource("GPIB::5::INSTR") client = TofuPilotClient() def run_hipot_test(serial, test_voltage_v=1500, duration_s=60, leakage_limit_ma=5.0): # Configure hipot tester hipot.write(f"VOLT {test_voltage_v}") hipot.write(f"TIME {duration_s}") hipot.write(f"CURR:LIM {leakage_limit_ma}") hipot.write("RAMP 2") # 2 second ramp # Run test hipot.write("TEST") hipot.query("*OPC?") # Wait for completion # Read results result = hipot.query("MEAS:RES?") # "PASS" or "FAIL" leakage = float(hipot.query("MEAS:CURR?")) * 1000 # Convert to mA passed = result.strip() == "PASS" client.create_run( procedure_id="HIPOT-SAFETY-TEST", unit_under_test={"serial_number": serial}, run_passed=passed, steps=[{ "name": "Dielectric Withstand", "step_type": "measurement", "status": passed, "measurements": [ {"name": "test_voltage_v", "value": test_voltage_v, "unit": "V"}, {"name": "duration_s", "value": duration_s, "unit": "s"}, {"name": "leakage_current_ma", "value": leakage, "unit": "mA", "limit_high": leakage_limit_ma}, {"name": "breakdown", "value": 0 if passed else 1, "unit": "bool", "limit_high": 0}, ], }], ) hipot.write("VOLT 0") # Discharge return passed ``` ### Multi-Point Hipot Testing Some products require hipot tests between multiple isolation boundaries. ```python filename="multi_point_hipot.py" # Test multiple isolation boundaries isolation_tests = [ {"name": "Line-to-Ground", "voltage": 1500, "limit_ma": 5.0}, {"name": "Line-to-Secondary", "voltage": 3000, "limit_ma": 5.0}, {"name": "Secondary-to-Ground", "voltage": 500, "limit_ma": 1.0}, ] steps = [] for test in isolation_tests: # Configure and run each test leakage = run_hipot_measurement(test["voltage"]) passed = leakage < test["limit_ma"] steps.append({ "name": test["name"], "step_type": "measurement", "status": passed, "measurements": [ {"name": f"voltage_{test['name'].lower().replace('-', '_')}", "value": test["voltage"], "unit": "V"}, {"name": f"leakage_{test['name'].lower().replace('-', '_')}", "value": leakage, "unit": "mA", "limit_high": test["limit_ma"]}, ], }) all_pass = all(s["status"] for s in steps) client.create_run( procedure_id="HIPOT-MULTI-POINT", unit_under_test={"serial_number": serial}, run_passed=all_pass, steps=steps, ) ``` ## Leakage Current Trending Even for passing units, track leakage current over production. A healthy process shows consistent leakage values. Changes indicate: | Trend | Possible cause | |-------|---------------| | Gradual increase | Contamination buildup on PCB, flux residue | | Step increase | New board revision, different conformal coating | | High variance | Inconsistent manufacturing (solder splash, debris) | | Bimodal distribution | Two different PCB suppliers or manufacturing lines | TofuPilot's measurement histogram for leakage current shows these patterns across your production. ## Compliance Records Safety agencies require per-unit hipot test records. TofuPilot stores: - Serial number of the tested unit - Test voltage applied - Duration of the test - Measured leakage current - Pass/fail result - Timestamp - Station/tester identification When an auditor asks for hipot records for a specific serial number or date range, pull them from TofuPilot in seconds. ## Safety Considerations Hipot testing involves lethal voltages. Always: - Use proper safety interlocks on the test fixture - Ensure the DUT is fully discharged after the test - Never bypass safety mechanisms on the hipot tester - Log test equipment calibration dates - Train operators on hipot safety procedures TofuPilot is the data management layer. The safety of the test setup is your responsibility. ## Ground Continuity Testing Ground continuity is often tested alongside hipot. It verifies that the safety ground connection has low enough resistance to carry fault current. ```python filename="ground_continuity.py" # Ground continuity test (typically 25A for 2 seconds) ground_resistance = measure_ground_resistance(current_a=25, duration_s=2) client.create_run( procedure_id="GROUND-CONTINUITY", unit_under_test={"serial_number": serial}, run_passed=ground_resistance < 0.1, steps=[{ "name": "Ground Bond", "step_type": "measurement", "status": ground_resistance < 0.1, "measurements": [ {"name": "ground_resistance_ohm", "value": ground_resistance, "unit": "ohm", "limit_high": 0.1}, {"name": "test_current_a", "value": 25, "unit": "A"}, ], }], ) ``` Most safety test workflows run ground continuity first, then hipot. If the ground bond is bad, there's no point running the hipot test. TofuPilot links both results to the same serial number for a complete safety test record. ### DO-178C Test Traceability with TofuPilot URL: https://www.tofupilot.com/guides/do-178c-test-traceability-with-tofupilot Learn how to maintain DO-178C compliant test traceability for airborne software using TofuPilot's structured test records and requirement mapping. # DO-178C Test Traceability with TofuPilot DO-178C (Software Considerations in Airborne Systems and Equipment Certification) requires complete traceability from requirements to test cases to test results. For hardware-software integrated systems, this means linking every test measurement to the requirement it verifies. TofuPilot provides the structured test data layer. ## DO-178C and Testing DO-178C defines five Design Assurance Levels (DAL), from A (catastrophic) to E (no safety effect). Higher levels require more rigorous testing and documentation. | DAL | Failure condition | Testing rigor | |-----|-------------------|---------------| | A | Catastrophic | Full MC/DC coverage, independence | | B | Hazardous | Full decision coverage | | C | Major | Full statement coverage | | D | Minor | Basic testing | | E | No effect | Minimal | Regardless of DAL, all levels require traceability between requirements, test cases, and test results. ## The Traceability Chain ``` Requirements → Test Cases → Test Procedures → Test Results ↓ ↓ ↓ ↓ DOORS Test Plan TofuPilot TofuPilot or Jama (doc) Procedure Run Results ``` TofuPilot handles the right half: test procedures (defined as procedure IDs with steps and measurements) and test results (actual run data with pass/fail). ## Mapping Requirements to Test Procedures Use a consistent naming convention that links TofuPilot procedures to requirements. ```python filename="do178c_test.py" from tofupilot import TofuPilotClient client = TofuPilotClient() # Procedure ID includes the requirement reference client.create_run( procedure_id="TC-SW-REQ-042-AIRSPEED-COMPUTATION", unit_under_test={ "serial_number": "ADC-UNIT-007", "part_number": "AIR-DATA-COMPUTER-V2", }, run_passed=True, steps=[{ "name": "Airspeed Computation Accuracy", "step_type": "measurement", "status": True, "measurements": [ {"name": "indicated_airspeed_error_kts", "value": 0.8, "unit": "kts", "limit_high": 2.0}, {"name": "true_airspeed_error_kts", "value": 1.2, "unit": "kts", "limit_high": 3.0}, {"name": "mach_number_error", "value": 0.002, "unit": "Mach", "limit_high": 0.005}, ], }, { "name": "Airspeed Range Verification", "step_type": "measurement", "status": True, "measurements": [ {"name": "min_airspeed_kts", "value": 30, "unit": "kts", "limit_high": 40}, {"name": "max_airspeed_kts", "value": 450, "unit": "kts", "limit_low": 400}, ], }], ) ``` ### Requirements Traceability Matrix | Req ID | Requirement | Test Procedure | TofuPilot Measurement | Limit | |--------|------------|----------------|----------------------|-------| | SW-REQ-042 | Airspeed computation accuracy < 2 kts | TC-SW-REQ-042 | indicated_airspeed_error_kts | < 2.0 kts | | SW-REQ-043 | True airspeed accuracy < 3 kts | TC-SW-REQ-042 | true_airspeed_error_kts | < 3.0 kts | | SW-REQ-044 | Mach computation accuracy < 0.005 | TC-SW-REQ-042 | mach_number_error | < 0.005 | | SW-REQ-045 | Operate from 40 to 400 kts | TC-SW-REQ-042 | min/max_airspeed_kts | 40/400 | ## Hardware-Software Integration Testing DO-178C Section 6.4 covers hardware/software integration testing. These tests verify that the software works correctly on the target hardware. ```python filename="hwsw_integration.py" # Hardware/Software integration test client.create_run( procedure_id="HWSW-INT-ADC-ARINC429", unit_under_test={"serial_number": "ADC-UNIT-007"}, run_passed=True, steps=[{ "name": "ARINC 429 Output Verification", "step_type": "measurement", "status": True, "measurements": [ {"name": "label_airspeed_rate_hz", "value": 25.0, "unit": "Hz", "limit_low": 24.5, "limit_high": 25.5}, {"name": "label_altitude_rate_hz", "value": 12.5, "unit": "Hz", "limit_low": 12.0, "limit_high": 13.0}, {"name": "ssm_status_normal", "value": 1, "unit": "bool", "limit_low": 1}, {"name": "data_latency_ms", "value": 18.2, "unit": "ms", "limit_high": 40.0}, ], }, { "name": "Watchdog Timer Verification", "step_type": "measurement", "status": True, "measurements": [ {"name": "watchdog_timeout_ms", "value": 50, "unit": "ms", "limit_low": 45, "limit_high": 55}, {"name": "reset_recovery_ms", "value": 120, "unit": "ms", "limit_high": 200}, ], }], ) ``` ## Regression Testing When software is modified, DO-178C requires regression testing to verify no unintended effects. TofuPilot makes regression tracking straightforward: 1. Run the full test suite on the new software version 2. Compare results against the baseline (previous version) in TofuPilot 3. Flag any measurements that changed beyond expected tolerance ```python filename="regression_check.py" # Compare test results across software versions baseline_runs = client.get_runs( procedure_id="TC-SW-REQ-042-AIRSPEED-COMPUTATION", limit=10, ) # Filter by software version through unit metadata or date range # Compare measurement values between versions ``` If a measurement that was 0.8 kts error in version 2.1 becomes 1.9 kts error in version 2.2, the regression test catches it before certification submission. ## DER/Auditor Evidence Package When presenting test evidence to a DER (Designated Engineering Representative) or certification authority: | Document | Source | |----------|--------| | Requirements Traceability Matrix | Your requirements tool + TofuPilot procedure mapping | | Test Procedures | TofuPilot procedure definitions with steps and limits | | Test Results | TofuPilot run data with measurements and pass/fail | | Test Coverage Analysis | Map of requirements to TofuPilot procedures | | Regression Test Report | TofuPilot comparison between software versions | TofuPilot provides the structured, timestamped test evidence. Your certification package references TofuPilot data as the authoritative test record. ## DO-254 Hardware Testing For DO-254 (hardware assurance), the same traceability principles apply. Hardware requirements map to hardware test procedures, which map to test results in TofuPilot. Common DO-254 test types tracked in TofuPilot: - FPGA functional verification - Environmental qualification - EMI/EMC compliance - Power supply characterization - Timing and performance verification Use the same procedure naming convention (requirement ID in the procedure ID) for consistent traceability across hardware and software testing. ### Burn-In Testing for Electronics URL: https://www.tofupilot.com/guides/burn-in-testing-for-electronics-a-complete-guide Learn how to set up burn-in tests for electronic assemblies, log results to TofuPilot, and use burn-in data to catch infant mortality defects. # Burn-In Testing for Electronics: A Complete Guide Burn-in testing operates products under stress for an extended period to catch infant mortality failures before they reach customers. It's the hardware equivalent of "let it run overnight and see if it crashes." TofuPilot tracks burn-in results alongside your other production tests for complete unit traceability. ## What Burn-In Testing Is The bathtub curve describes electronic failure rates over time: 1. **Infant mortality** (early life): High failure rate, decreasing. Caused by manufacturing defects, weak solder joints, marginal components. 2. **Useful life**: Low, constant failure rate. 3. **Wear-out**: Increasing failure rate at end of life. Burn-in targets the infant mortality region. By operating the product under stress for hours or days, you accelerate the aging process and force weak units to fail in the factory instead of in the field. ## Burn-In Parameters | Parameter | Typical range | Purpose | |-----------|--------------|---------| | Temperature | 55-85°C (elevated) | Accelerate thermally-activated failures | | Voltage | 10-20% above nominal | Stress power supply and regulators | | Duration | 24-168 hours | Enough time for weak units to fail | | Monitoring interval | Every 1-4 hours | Catch failures as they happen | | Functional check | Before and after, plus periodic | Verify unit still works | ## Setting Up Burn-In with TofuPilot ### Pre-Burn-In Functional Test Always run a functional test before burn-in to establish a baseline. ```python filename="burn_in_workflow.py" from tofupilot import TofuPilotClient client = TofuPilotClient() def pre_burn_in_test(serial): vcc = measure_voltage() current = measure_current() temp = measure_temperature() client.create_run( procedure_id="BURN-IN-PRE-CHECK", unit_under_test={"serial_number": serial}, run_passed=True, steps=[{ "name": "Pre-Burn-In Baseline", "step_type": "measurement", "status": True, "measurements": [ {"name": "vcc_3v3", "value": vcc, "unit": "V", "limit_low": 3.25, "limit_high": 3.35}, {"name": "current_ma", "value": current * 1000, "unit": "mA", "limit_low": 30, "limit_high": 60}, {"name": "board_temp_c", "value": temp, "unit": "°C"}, ], }], ) ``` ### Periodic Monitoring During Burn-In Log measurements at regular intervals during the burn-in period. ```python filename="burn_in_monitor.py" import time def burn_in_monitor(serial, duration_hours=48, check_interval_hours=4): checks = int(duration_hours / check_interval_hours) for check in range(1, checks + 1): elapsed = check * check_interval_hours # Measure without removing unit from burn-in chamber vcc = measure_voltage() current = measure_current() temp = measure_temperature() passed = 3.20 <= vcc <= 3.40 and current * 1000 <= 80 client.create_run( procedure_id="BURN-IN-MONITOR", unit_under_test={"serial_number": serial}, run_passed=passed, steps=[{ "name": f"Hour {elapsed} Check", "step_type": "measurement", "status": passed, "measurements": [ {"name": "hours_elapsed", "value": elapsed, "unit": "h"}, {"name": "vcc_3v3", "value": vcc, "unit": "V", "limit_low": 3.20, "limit_high": 3.40}, {"name": "current_ma", "value": current * 1000, "unit": "mA", "limit_high": 80}, {"name": "chamber_temp_c", "value": temp, "unit": "°C"}, ], }], ) if not passed: print(f"FAIL at hour {elapsed}: {serial}") return False time.sleep(check_interval_hours * 3600) return True ``` ### Post-Burn-In Functional Test After burn-in, run the same functional test as before. Compare pre and post measurements. ```python filename="post_burn_in.py" def post_burn_in_test(serial): vcc = measure_voltage() current = measure_current() client.create_run( procedure_id="BURN-IN-POST-CHECK", unit_under_test={"serial_number": serial}, run_passed=True, steps=[{ "name": "Post-Burn-In Verification", "step_type": "measurement", "status": True, "measurements": [ {"name": "vcc_3v3", "value": vcc, "unit": "V", "limit_low": 3.25, "limit_high": 3.35}, {"name": "current_ma", "value": current * 1000, "unit": "mA", "limit_low": 30, "limit_high": 60}, ], }], ) ``` ## Analyzing Burn-In Data ### Pre vs. Post Comparison TofuPilot lets you compare pre-burn-in and post-burn-in measurements for the same unit. Look for: | Change | What it means | |--------|--------------| | No change | Unit is stable, burn-in passed | | Small shift (< 1%) | Normal thermal aging, acceptable | | Large shift (> 5%) | Component degradation, investigate | | Failure during burn-in | Infant mortality defect caught | ### Burn-In Failure Rate Tracking Track the percentage of units that fail during burn-in over time. | Month | Units burned in | Failures | Burn-in failure rate | |-------|----------------|----------|---------------------| | Jan | 1,000 | 8 | 0.8% | | Feb | 1,200 | 12 | 1.0% | | Mar | 1,100 | 25 | 2.3% | If the burn-in failure rate increases, something changed in your manufacturing process. The burn-in is doing its job by catching these defects, but you need to find and fix the root cause. ### When to Skip Burn-In Burn-in costs time and money. Not every product needs it. | Factor | Burn-in recommended | Burn-in optional | |--------|--------------------|-----------------| | Safety-critical product | Yes | | | High field failure cost | Yes | | | Mature, high-yield process | | Yes | | Consumer electronics (cost-sensitive) | | Yes | | Medical/aerospace/defense | Yes | | Use TofuPilot data to make this decision. If burn-in catches 2% of units and field failure costs $5,000 per incident, burn-in pays for itself. If burn-in catches 0.01% and field failure costs $50, it doesn't. ## Burn-In Rack Monitoring For high-volume burn-in, you may have racks with dozens of units burning in simultaneously. Monitor them all through TofuPilot. Each slot in the rack uploads periodic measurements. The dashboard shows: - Which slots currently have units - Current temperature and power consumption per slot - Any failures that occurred during the burn-in period - Historical burn-in yield per rack position (catches rack-specific issues like bad power connections) If rack position 12 has a higher failure rate than other positions, the rack needs maintenance, not the product. ### Build a Device History Record URL: https://www.tofupilot.com/guides/how-to-build-a-device-history-record-with-tofupilot Learn how to compile FDA-compliant Device History Records using TofuPilot's per-unit test data, traceability, and structured measurement storage. # How to Build a Device History Record with TofuPilot The Device History Record (DHR) is a mandatory document for FDA-regulated medical devices. It's the complete production record for each finished device: what was built, how it was tested, and what the results were. TofuPilot provides the test data component of the DHR automatically. ## What the FDA Requires 21 CFR 820.184 (Device History Record) requires documentation of: | DHR component | What it includes | TofuPilot's role | |--------------|-----------------|------------------| | Production dates | When the device was manufactured | Test run timestamps | | Quantity manufactured | How many in the lot/batch | Run count by batch | | Acceptance records | Test results showing conformity | All measurements with limits | | Primary identification label | Serial number, UDI | Unit under test serial | | Labeling | Labels used on the device | (Outside TofuPilot scope) | | Equipment used | Test stations, calibration status | Station identification | ## Test Data as DHR Evidence Every test run in TofuPilot is a DHR evidence record. For each device serial number, TofuPilot stores: ```python filename="dhr_test_record.py" from tofupilot import TofuPilotClient client = TofuPilotClient() # ICT (In-Circuit Test) client.create_run( procedure_id="ICT-MEDICAL-BOARD", unit_under_test={ "serial_number": "MD-2025-00421", "part_number": "CARDIAC-MONITOR-PCB-R5", }, run_passed=True, steps=[{ "name": "Component Verification", "step_type": "measurement", "status": True, "measurements": [ {"name": "r1_resistance_kohm", "value": 10.02, "unit": "kohm", "limit_low": 9.5, "limit_high": 10.5}, {"name": "c1_capacitance_uf", "value": 4.72, "unit": "uF", "limit_low": 4.23, "limit_high": 5.17}, {"name": "u1_continuity", "value": 1, "unit": "bool", "limit_low": 1}, ], }], ) # Functional test client.create_run( procedure_id="FUNC-CARDIAC-MONITOR", unit_under_test={"serial_number": "MD-2025-00421"}, run_passed=True, steps=[{ "name": "ECG Signal Chain", "step_type": "measurement", "status": True, "measurements": [ {"name": "ecg_gain_db", "value": 60.2, "unit": "dB", "limit_low": 59.0, "limit_high": 61.0}, {"name": "ecg_cmrr_db", "value": 112, "unit": "dB", "limit_low": 100}, {"name": "ecg_noise_uv_rms", "value": 8.3, "unit": "uV", "limit_high": 15}, ], }], ) # Electrical safety client.create_run( procedure_id="SAFETY-IEC60601", unit_under_test={"serial_number": "MD-2025-00421"}, run_passed=True, steps=[{ "name": "Patient Leakage", "step_type": "measurement", "status": True, "measurements": [ {"name": "patient_leakage_normal_ua", "value": 4.2, "unit": "uA", "limit_high": 10}, {"name": "patient_leakage_sfc_ua", "value": 22.1, "unit": "uA", "limit_high": 50}, ], }], ) # Final inspection client.create_run( procedure_id="FINAL-INSPECTION", unit_under_test={"serial_number": "MD-2025-00421"}, run_passed=True, steps=[{ "name": "Cosmetic and Label Check", "step_type": "measurement", "status": True, "measurements": [ {"name": "label_present", "value": 1, "unit": "bool", "limit_low": 1}, {"name": "cosmetic_pass", "value": 1, "unit": "bool", "limit_low": 1}, {"name": "udi_barcode_readable", "value": 1, "unit": "bool", "limit_low": 1}, ], }], ) ``` ## Compiling the DHR Search by serial number in TofuPilot to retrieve all test records for a device. For device `MD-2025-00421`, the DHR test section shows: | Test | Date | Result | Station | |------|------|--------|---------| | ICT-MEDICAL-BOARD | 2025-03-15 09:22 | PASS | STN-ICT-02 | | FUNC-CARDIAC-MONITOR | 2025-03-15 10:15 | PASS | STN-FUNC-01 | | SAFETY-IEC60601 | 2025-03-15 11:30 | PASS | STN-SAFETY-01 | | FINAL-INSPECTION | 2025-03-15 14:00 | PASS | STN-FINAL-01 | Each row links to the complete measurement data: every value, every limit, every pass/fail status. ## Handling Nonconformities in the DHR When a device fails a test and is reworked, the DHR must show: 1. The original failure (which measurement, what value) 2. The disposition decision (rework, scrap, use-as-is) 3. The retest result after rework TofuPilot captures #1 and #3 automatically. For #2, document the disposition in your QMS and cross-reference the TofuPilot run ID. ```python filename="dhr_retest.py" # Original test failed # Run ID: run-001 (patient leakage at 12.3 uA, limit 10 uA) # After rework (replaced connector), retest client.create_run( procedure_id="SAFETY-IEC60601-RETEST", unit_under_test={"serial_number": "MD-2025-00421"}, run_passed=True, steps=[{ "name": "Patient Leakage (Post-Rework)", "step_type": "measurement", "status": True, "measurements": [ {"name": "patient_leakage_normal_ua", "value": 3.8, "unit": "uA", "limit_high": 10}, ], }], ) ``` The DHR for this device now shows: original fail, rework disposition (from QMS), and passing retest. ## Batch DHR Reports For batch release, pull all devices in a production lot and verify complete test records. ```python filename="batch_dhr_check.py" from tofupilot import TofuPilotClient client = TofuPilotClient() # Required procedures for a complete DHR required_procedures = [ "ICT-MEDICAL-BOARD", "FUNC-CARDIAC-MONITOR", "SAFETY-IEC60601", "FINAL-INSPECTION", ] # Check all devices in the batch batch_serials = [f"MD-2025-{i:05d}" for i in range(400, 450)] incomplete = [] for serial in batch_serials: runs = client.get_runs(unit_serial=serial) completed_procedures = {r["procedure_id"] for r in runs if r["run_passed"]} missing = set(required_procedures) - completed_procedures if missing: incomplete.append({"serial": serial, "missing": missing}) if incomplete: print(f"{len(incomplete)} devices have incomplete DHRs:") for d in incomplete: print(f" {d['serial']}: missing {d['missing']}") else: print("All devices have complete DHRs. Batch ready for release.") ``` ## Retention and Retrieval FDA requires DHR retention for the lifetime of the device plus additional years. TofuPilot stores all test data with timestamps and audit trails. Configure your data retention policy to match your regulatory requirements. When FDA asks for a specific device's DHR during an inspection, search by serial number and export the complete test history. Structured, timestamped, immutable records replace paper binders and Excel files. ## Integration with Your QMS TofuPilot handles the test data component of the DHR. Your QMS (quality management system) handles: - Bill of materials and component traceability - Manufacturing work orders - Rework and disposition records - Labeling and packaging records - Release and distribution records Together, TofuPilot and your QMS provide the complete DHR. The key is using consistent serial numbers across both systems so records can be cross-referenced. ### ISO 13485 Test Data Requirements with TofuPilot URL: https://www.tofupilot.com/guides/iso-13485-test-data-requirements-with-tofupilot Learn how to meet ISO 13485 test documentation requirements for medical devices using TofuPilot's structured data and traceability features. # ISO 13485 Test Data Requirements with TofuPilot ISO 13485 is the quality management standard for medical device manufacturers. It requires rigorous control of production processes, including complete test records for every device produced. TofuPilot provides the structured, traceable test data storage that ISO 13485 demands. ## What ISO 13485 Requires for Test Data ISO 13485 Section 7.5.1 (Control of Production and Service Provision) and Section 8.2.4 (Monitoring and Measurement of Product) require: | Requirement | ISO 13485 clause | What it means for testing | |------------|------------------|--------------------------| | Process validation | 7.5.2 | Prove your test process produces consistent results | | Identification and traceability | 7.5.3 | Link every test result to a specific device serial number | | Monitoring and measurement | 8.2.4 | Define acceptance criteria and record conformity evidence | | Control of records | 4.2.5 | Maintain test records, ensure they're retrievable and protected | | Nonconforming product | 8.3 | Document what happens to devices that fail testing | ## How TofuPilot Maps to ISO 13485 ### Traceability (7.5.3) Every test run in TofuPilot is linked to a unit serial number. Search by serial to see the complete test history of any device. ```python filename="medical_device_test.py" from tofupilot import TofuPilotClient client = TofuPilotClient() # Every run ties to a unique device identifier (UDI) client.create_run( procedure_id="FINAL-INSPECTION-IEC60601", unit_under_test={ "serial_number": "MD-2025-00421", # UDI or serial "part_number": "PULSE-OX-V3", }, run_passed=True, steps=[{ "name": "Electrical Safety", "step_type": "measurement", "status": True, "measurements": [ {"name": "earth_leakage_ua", "value": 287, "unit": "uA", "limit_high": 500}, {"name": "patient_leakage_ua", "value": 8.3, "unit": "uA", "limit_high": 10}, {"name": "insulation_resistance_mohm", "value": 520, "unit": "Mohm", "limit_low": 100}, ], }, { "name": "Functional Performance", "step_type": "measurement", "status": True, "measurements": [ {"name": "spo2_accuracy_pct", "value": 1.2, "unit": "%", "limit_high": 2.0}, {"name": "heart_rate_accuracy_bpm", "value": 1.5, "unit": "BPM", "limit_high": 3.0}, ], }], ) ``` ### Acceptance Criteria (8.2.4) ISO 13485 requires documented acceptance criteria for every inspection and test. In TofuPilot, limits on measurements serve as your acceptance criteria. | Measurement | Acceptance criteria | Standard reference | |------------|--------------------|--------------------| | Earth leakage | < 500 uA | IEC 60601-1 | | Patient leakage | < 10 uA (Type BF) | IEC 60601-1 | | Insulation resistance | > 100 Mohm | IEC 60601-1 | | SpO2 accuracy | < 2% ARMS | ISO 80601-2-61 | These limits are defined in the test procedure and enforced by TofuPilot on every run. ### Control of Records (4.2.5) ISO 13485 requires records to be: | Requirement | How TofuPilot meets it | |------------|----------------------| | Legible | Structured data, not handwritten | | Readily identifiable | Searchable by serial, procedure, date | | Retrievable | API and dashboard access | | Protected from damage | Cloud-hosted, backed up | | Protected from loss | Redundant storage | | Retained for defined period | Data retention per your QMS policy | ### Nonconforming Product (8.3) When a device fails testing, ISO 13485 requires documentation of: 1. The nature of the nonconformity (which measurement failed) 2. Actions taken (rework, scrap, use-as-is disposition) 3. Re-inspection results after rework TofuPilot captures #1 automatically (failed measurements with values and limits). For #2 and #3, the retest flow creates additional run records linked to the same serial number. ```python filename="retest_after_rework.py" # Device failed initial test # After rework, retest and record client.create_run( procedure_id="FINAL-INSPECTION-IEC60601-RETEST", unit_under_test={ "serial_number": "MD-2025-00421", }, run_passed=True, steps=[{ "name": "Post-Rework Electrical Safety", "step_type": "measurement", "status": True, "measurements": [ {"name": "earth_leakage_ua", "value": 312, "unit": "uA", "limit_high": 500}, {"name": "patient_leakage_ua", "value": 7.1, "unit": "uA", "limit_high": 10}, ], }], ) ``` The device now has two test records in TofuPilot: the initial failure and the post-rework pass. Both are traceable, timestamped, and immutable. ## Device History Record (DHR) The DHR is a compilation of records that documents the production history of a finished device. Test records are a key component. TofuPilot contributes to the DHR by providing: - All test procedures executed on the device - All measurement results with pass/fail status - Dates and times of each test - Which station/tester ran each test - Any retests after rework Pull a device's complete test history by serial number for DHR compilation. ## Audit Readiness When an auditor asks for test records, you need to produce them quickly and completely. Common audit requests and how to answer them with TofuPilot: | Auditor request | TofuPilot response | |----------------|-------------------| | "Show me test records for lot 2025-03" | Filter by date range, export all runs | | "What's the acceptance criteria for leakage testing?" | Show measurement limits in the procedure | | "Show me all devices that failed and were reworked" | Filter for failed runs, check retests by serial | | "Prove your test equipment is calibrated" | Station metadata with calibration dates | | "What's your first-pass yield for this product?" | Procedure dashboard FPY metric | ## IEC 60601 Electrical Safety Tests Medical devices connected to patients require IEC 60601-1 electrical safety testing. Key measurements: | Test | Type BF limit | Type CF limit | |------|--------------|---------------| | Earth leakage (normal) | 500 uA | 500 uA | | Earth leakage (single fault) | 1000 uA | 1000 uA | | Patient leakage (normal) | 100 uA | 10 uA | | Patient leakage (single fault) | 500 uA | 50 uA | Log all of these in TofuPilot with limits matching your device classification. The structured records serve as evidence of conformity for regulatory submissions. ## Software Validation If you're using TofuPilot as part of your quality system, ISO 13485 Section 7.5.2.1 requires validation of software used in production and quality management. Work with your quality team to document TofuPilot's role in your QMS and include it in your software validation plan. ### What Is an AI-Native Test Station URL: https://www.tofupilot.com/guides/what-is-an-ai-native-test-station An AI-native test station is built around data and inference from the start, not bolted on after. Learn what it means and how it changes manufacturing test. # What Is an AI-Native Test Station An AI-native test station is designed from the ground up with data collection, real-time analytics, and machine learning as core capabilities, not as add-ons. The difference is architectural: instead of running tests and exporting CSVs for offline analysis, an AI-native station streams structured data, learns from every run, and adapts its behavior based on what the data shows. ## Traditional vs AI-Native | Aspect | Traditional Station | AI-Native Station | |--------|-------------------|-------------------| | Data flow | Test runs, results saved to local file or database | Test runs, results stream to analytics platform in real time | | Data format | CSV, proprietary binary, or unstructured logs | Structured measurements with units, limits, and metadata | | Analytics | Offline, batch processing, manual export | Real-time dashboards, automated trend detection | | Limit management | Hard-coded in test script, changed manually | Informed by production data, adjusted based on distributions | | Failure response | Engineer reviews logs after the fact | Immediate alerts, automated root cause suggestions | | Cross-station learning | Each station is isolated | All stations share data, models improve from fleet-wide patterns | | Test sequence | Static, same for every unit | Can adapt based on upstream data or risk score | ## The Three Pillars ### 1. Structured Data by Default Every measurement has a name, value, unit, and limits. Every run has a serial number, timestamp, station ID, and pass/fail result. This isn't optional. It's the foundation. ```python filename="ai_native_test.py" import openhtf as htf from openhtf.util import units @htf.measures( htf.Measurement("supply_voltage_V") .in_range(minimum=4.9, maximum=5.1) .with_units(units.VOLT), htf.Measurement("boot_time_ms") .in_range(maximum=2000) .with_units(units.MILLISECOND), ) def phase_functional_check(test): """Every measurement is structured and traceable.""" test.measurements.supply_voltage_V = 5.01 test.measurements.boot_time_ms = 1280 ``` Without structured data, AI has nothing to learn from. The most common reason AI projects fail in manufacturing is not bad algorithms. It's bad data. ### 2. Real-Time Streaming Test results flow to a central platform as they happen, not at the end of the shift or when someone remembers to export. ```python filename="ai_native_test.py" from tofupilot.openhtf import TofuPilot test = htf.Test(phase_functional_check) with TofuPilot(test): test.execute(test_start=lambda: input("Scan serial: ")) ``` Real-time streaming enables: - Immediate yield alerts when failures spike - Live measurement distributions across all stations - Cross-station comparison to detect fixture or equipment issues - Operator UI that shows results as they happen ### 3. Feedback Loops The station learns from its own data. Historical results inform future decisions: | Feedback Loop | What It Does | |--------------|-------------| | Limit refinement | Production data shows the real distribution, limits get tightened or relaxed | | Failure prioritization | Pareto analysis ranks which failures matter most | | Station health monitoring | Throughput and yield trends detect equipment degradation | | Measurement drift detection | Control charts flag when a parameter starts trending | ## What Changes in Practice ### For Test Engineers | Before | After | |--------|-------| | Write test, deploy, forget | Write test, deploy, monitor, iterate | | Set limits from datasheet | Set initial limits, refine from production data | | Debug failures from logs | Query failure patterns across thousands of runs | | Optimize one station at a time | Compare performance across all stations instantly | ### For Operators | Before | After | |--------|-------| | Run test, read pass/fail on terminal | Run test, see results on streaming operator UI | | Report failures verbally | Failures are logged automatically with full context | | No visibility into trends | Dashboard shows yield and throughput in real time | ### For Quality Engineers | Before | After | |--------|-------| | Export CSVs, build reports in Excel | Reports generated from live data | | Monthly quality reviews with stale data | Real-time quality dashboards | | Root cause analysis takes weeks | Failure correlations visible immediately | ## Building an AI-Native Station You don't need to buy new hardware. An AI-native station is a software architecture choice: | Component | Traditional | AI-Native | |-----------|-----------|-----------| | Test framework | Any (OpenHTF, pytest, custom) | Same, but with structured measurements | | Data backend | Local files, local database | Cloud or self-hosted analytics platform | | Operator interface | Terminal or custom GUI | Web-based streaming UI | | Analytics | Excel, manual reports | Automated dashboards, alerts, trend detection | | Integration | Point-to-point, custom scripts | API-based, standard data formats | The key insight: making a station AI-native is not about adding AI features. It's about structuring the data and infrastructure so that AI features become possible. Get the data right, and the intelligence follows. ### HIL vs SIL vs MIL Testing: When to Use Each URL: https://www.tofupilot.com/guides/hil-vs-sil-vs-mil-testing-when-to-use-each-with-tofupilot Compare MIL, SIL, and HIL testing approaches with cost, speed, and fidelity tradeoffs, plus Python examples for hardware-in-the-loop validation with TofuPilot. MIL, SIL, and HIL are three levels of hardware-in-the-loop testing that trade cost and speed for fidelity. Use MIL during algorithm development, SIL to validate compiled code, and HIL to verify behavior against real hardware before production. ## What MIL, SIL, and HIL Mean **Model-in-the-Loop (MIL)** runs your control algorithm as a simulation model against a simulated plant. No hardware involved. Fast iteration, low cost, but lowest fidelity. **Software-in-the-Loop (SIL)** compiles your production code and runs it on a host machine against a simulated environment. Same binary, no hardware. Catches code generation bugs MIL misses. **Hardware-in-the-Loop (HIL)** runs your production code on the target ECU or microcontroller, connected to a real-time simulator that emulates the physical environment. Highest fidelity, highest cost. ## Comparison Matrix | Dimension | MIL | SIL | HIL | |-----------|-----|-----|-----| | Cost | Low | Low-medium | High | | Speed | Fast (seconds) | Medium (minutes) | Slow (minutes-hours) | | Fidelity | Low | Medium | High | | Hardware required | None | None | Target ECU + simulator rack | | Typical use | Algorithm design | Code verification | System validation | | CI-friendly | Yes | Yes | No (dedicated bench) | | Bug discovery | Design errors | Codegen errors | Integration, timing | ## When to Use Each ### MIL: Early Algorithm Development Use MIL when the algorithm is still changing, you need rapid iteration, hardware is not yet available, or you're running parametric sweeps. MIL is wrong when you need to validate timing behavior, interrupt latency, or bus communication. ### SIL: Code Verification Before Hardware Use SIL when code generation is complete, you want to catch regressions in CI, you're validating numerical equivalence between model and generated code, or hardware bench slots are scarce. SIL misses real-time timing faults and hardware-specific peripheral behavior. ### HIL: Pre-Production System Validation Use HIL when the ECU firmware is frozen, you need to validate timing and bus behavior, certification requires hardware evidence, or you're running overnight regression suites. HIL is expensive to set up and slow to iterate. ## Example: Voltage Limit Test at Each Level The same test logic runs at all three levels. Only the plug changes. ### MIL: Simulated Sensor ```python filename="tests/mil/test_voltage_limit.py" import openhtf as htf from openhtf.util import units from tofupilot.openhtf import TofuPilot class SimulatedVoltageSensor(htf.plugs.BasePlug): """Returns a deterministic value from a simulation model.""" def setUp(self): pass def read_voltage(self) -> float: return 11.8 def tearDown(self): pass @htf.plug(sensor=SimulatedVoltageSensor) @htf.measures( htf.Measurement("battery_voltage") .in_range(minimum=10.0, maximum=14.5) .with_units(units.VOLT), ) def measure_battery_voltage(test, sensor): test.measurements.battery_voltage = sensor.read_voltage() def main(): test = htf.Test(measure_battery_voltage, test_name="battery-voltage-mil") with TofuPilot(test): test.execute(test_start=lambda: "UNIT-MIL-001") if __name__ == "__main__": main() ``` ### SIL: Compiled Code Against Simulated Bus ```python filename="tests/sil/test_voltage_limit.py" import ctypes import openhtf as htf from openhtf.util import units from tofupilot.openhtf import TofuPilot class SILVoltageSensor(htf.plugs.BasePlug): """Calls compiled production C library via ctypes.""" LIB_PATH = "./build/battery_monitor.so" def setUp(self): self._lib = ctypes.CDLL(self.LIB_PATH) self._lib.get_battery_voltage.restype = ctypes.c_float def read_voltage(self) -> float: return float(self._lib.get_battery_voltage()) def tearDown(self): pass @htf.plug(sensor=SILVoltageSensor) @htf.measures( htf.Measurement("battery_voltage") .in_range(minimum=10.0, maximum=14.5) .with_units(units.VOLT), ) def measure_battery_voltage(test, sensor): test.measurements.battery_voltage = sensor.read_voltage() def main(): test = htf.Test(measure_battery_voltage, test_name="battery-voltage-sil") with TofuPilot(test): test.execute(test_start=lambda: "UNIT-SIL-001") if __name__ == "__main__": main() ``` ### HIL: Real ECU on Hardware Bench ```python filename="tests/hil/test_voltage_limit.py" import can import openhtf as htf from openhtf.util import units from tofupilot.openhtf import TofuPilot class HILVoltageSensor(htf.plugs.BasePlug): """Reads voltage from real ECU over CAN bus.""" CHANNEL = "can0" BUSTYPE = "socketcan" def setUp(self): self._bus = can.interface.Bus(channel=self.CHANNEL, bustype=self.BUSTYPE) def read_voltage(self) -> float: msg = self._bus.recv(timeout=1.0) raw = msg.data[2] return raw * 0.1 def tearDown(self): self._bus.shutdown() @htf.plug(sensor=HILVoltageSensor) @htf.measures( htf.Measurement("battery_voltage") .in_range(minimum=10.0, maximum=14.5) .with_units(units.VOLT), ) def measure_battery_voltage(test, sensor): test.measurements.battery_voltage = sensor.read_voltage() def main(): test = htf.Test(measure_battery_voltage, test_name="battery-voltage-hil") with TofuPilot(test): test.execute(test_start=lambda: "UNIT-HIL-001") if __name__ == "__main__": main() ``` ## How TofuPilot Tracks Results Across All Levels Each test run uploads to TofuPilot with the same measurement name (`battery_voltage`) regardless of level. The `test_name` field distinguishes the environment. | test_name | Level | Traceability | |-----------|-------|-------------| | `battery-voltage-mil` | MIL | Algorithm baseline | | `battery-voltage-sil` | SIL | Codegen regression | | `battery-voltage-hil` | HIL | Hardware validation | A unit that passes MIL but fails SIL indicates a code generation issue. A unit that passes SIL but fails HIL indicates a hardware integration issue. ## Decision Framework | Activity | Recommended Level | |----------|-------------------| | Tuning control gains | MIL | | Regression testing in CI | SIL | | Verifying code generation output | SIL | | Validating CAN/LIN bus behavior | HIL | | Interrupt and timing verification | HIL | | Overnight fault injection suite | HIL | | Parametric sweep (100+ cases) | MIL or SIL | | Pre-release sign-off | HIL | | Root-cause analysis of field issue | HIL | ### Calculate ROI of Test Automation URL: https://www.tofupilot.com/guides/how-to-calculate-roi-of-test-automation-with-tofupilot Learn how to build a business case for hardware test automation and measure ROI using TofuPilot's yield, cycle time, and defect data. # How to Calculate ROI of Test Automation with TofuPilot Test automation costs money upfront: fixtures, instruments, software, integration time. But it saves money continuously: fewer field failures, faster throughput, less manual labor. TofuPilot gives you the data to quantify both sides of the equation. ## The ROI Framework ``` ROI = (Annual Savings - Annual Cost) / Initial Investment × 100% ``` The challenge isn't the formula. It's getting accurate numbers for savings and costs. TofuPilot provides the data for the savings side. ## Cost of NOT Automating ### Manual Testing Costs | Cost item | Manual test | Automated test | |-----------|------------|----------------| | Operator time per unit | 15-30 min | 1-3 min | | Data recording | Manual entry (error-prone) | Automatic | | Report generation | 3-4 hours/week | Zero (live dashboard) | | Debug time per failure | 2-4 hours | 15-30 min | | Operator training | Weeks | Days | ### Field Failure Costs This is usually the biggest number. A field failure costs 10-100x more than catching the same defect in the factory. | Stage where defect is caught | Typical cost | |------------------------------|-------------| | In-circuit test (ICT) | $1-5 per unit | | Functional test | $5-20 per unit | | Burn-in / ESS | $10-50 per unit | | Customer site (warranty) | $100-5,000 per unit | | Safety recall | $10,000+ per unit | ## Measuring Savings with TofuPilot Data ### Defect Escape Rate Track how many defects your test catches vs. how many reach customers. ```python filename="escape_rate.py" from tofupilot import TofuPilotClient client = TofuPilotClient() # Get production test data runs = client.get_runs( procedure_id="FINAL-FUNCTIONAL", limit=10000, ) total = len(runs) caught = sum(1 for r in runs if not r["run_passed"]) catch_rate = caught / total * 100 print(f"Units tested: {total}") print(f"Defects caught: {caught}") print(f"Catch rate: {catch_rate:.2f}%") print(f"Estimated escapes (if no test): {caught} defects reaching customers") print(f"Field failure cost avoided: ${caught * 500:,.0f}") # $500 per field failure ``` ### Yield Improvement Value Track FPY over time. Every percentage point of yield improvement has a dollar value. ``` Before TofuPilot: 94% FPY After TofuPilot: 97% FPY Improvement: 3 percentage points Annual production: 50,000 units Units saved from rework: 50,000 × 0.03 = 1,500 units Rework cost per unit: $25 Annual rework savings: 1,500 × $25 = $37,500 ``` ### Debug Time Reduction Track how long it takes to diagnose failures before and after TofuPilot. | Metric | Before | After | Savings | |--------|--------|-------|---------| | Avg debug time per failure | 3 hours | 0.5 hours | 2.5 hours | | Failures per week | 20 | 20 | - | | Weekly debug hours | 60 hours | 10 hours | 50 hours | | Engineer cost per hour | $75 | $75 | - | | Weekly savings | - | - | $3,750 | | Annual savings | - | - | $195,000 | ### Report Time Elimination | Metric | Before | After | Savings | |--------|--------|-------|---------| | Weekly report preparation | 4 hours | 0 hours | 4 hours | | Monthly quality review prep | 8 hours | 0 hours | 8 hours | | Annual report hours saved | 304 hours | 0 hours | 304 hours | | Cost saved | - | - | $22,800 | ## Building the Business Case ### Costs (Year 1) | Item | Cost | |------|------| | Test fixtures (2 stations) | $20,000 | | Test instruments | $30,000 | | TofuPilot subscription | $X,000/year | | Integration engineering (2 weeks) | $15,000 | | **Total Year 1** | **$65,000 + subscription** | ### Annual Savings | Savings category | Amount | |-----------------|--------| | Field failure prevention | $150,000 | | Debug time reduction | $195,000 | | Rework reduction (yield improvement) | $37,500 | | Report elimination | $22,800 | | Operator time reduction | $50,000 | | **Total annual savings** | **$455,300** | ### ROI Calculation ``` Year 1 ROI = ($455,300 - $65,000) / $65,000 × 100% = 600% Payback period = $65,000 / ($455,300 / 12) = 1.7 months ``` ## Using TofuPilot to Track ROI Over Time Once you're running with TofuPilot, track these metrics monthly: | KPI | How to measure | Target | |-----|---------------|--------| | First-pass yield | TofuPilot procedure dashboard | > 95% | | Defect escape rate | Field returns / units shipped | < 0.1% | | Mean time to diagnosis | Avg debug time per failure | < 30 min | | Test throughput | Units tested per shift | Increasing | | Rework rate | Failed units reworked / total | Decreasing | These KPIs make the ongoing ROI visible to management. When someone asks "Is the test system worth it?" point to the dashboard. ## Common Objections and Responses | Objection | Response | |-----------|----------| | "We don't have enough volume" | Calculate: how many field failures pay for the test system? | | "Manual testing is fine" | Ask: how many hours/week on reports, data gathering, debugging? | | "It's too expensive" | Compare to the cost of one product recall | | "We don't have time to set it up" | Start with one test, one station. Expand after proving value | ### Write a Functional Test Spec for PCBA URL: https://www.tofupilot.com/guides/how-to-write-a-functional-test-spec-for-pcba-with-tofupilot Learn how to build a functional test specification for PCBA production, map schematic nets to measurements with datasheet-derived limits, and translate to. A functional test specification defines every measurement your board must pass before it ships. It maps schematic nets to physical test points, sets pass/fail limits from datasheets, and becomes the contract between hardware design and production test. This guide shows you how to build that document and turn it into executable OpenHTF phases tracked in TofuPilot. ## What a Functional Test Spec Is A functional test spec is a structured table that answers three questions for every signal on your board: 1. What are you measuring? (net name, test point, measurement type) 2. What is the expected value? (nominal from schematic or datasheet) 3. What is the acceptable range? (min/max limits with tolerance) It is not a test script. It is the source of truth that drives the test script. ## Mapping Schematic Nets to Test Points Start with your schematic and export a net list. For each net that carries a testable signal, identify the physical access point on the board. | Net Name | Test Point | Signal Type | Nominal | Notes | |----------|-----------|------------|---------|-------| | VCC_3V3 | TP1 | DC voltage | 3.3 V | LDO output | | VCC_5V0 | TP2 | DC voltage | 5.0 V | USB VBUS | | XTAL_OUT | TP3 | Frequency | 12 MHz | Crystal oscillator | | I2C_SDA | TP4 | Logic high | 3.3 V | Pull-up to VCC_3V3 | | VBAT | TP5 | DC voltage | 3.7 V | Li-ion nominal | | LED_PWM | TP6 | Duty cycle | 50% | Default firmware state | Rules for choosing test points: - Prefer dedicated test vias over component pads - Include a ground reference TP near each measurement cluster - Number test points in physical order to reduce fixture wiring complexity ## Defining Limits from Datasheets Limits come from two sources: component datasheets and system-level margins. Trace every limit to a document and a line item. ### Voltage Regulators For an LDO like the TLV1117-3.3, the datasheet specifies: - Output voltage accuracy: +/-1% over temperature - Line regulation: 0.2% typical - Load regulation: 0.4% typical At 3.3 V nominal with +/-1% accuracy plus 0.4% load regulation: | Net | Nominal | Min | Max | Source | |-----|---------|-----|-----|--------| | VCC_3V3 | 3.300 V | 3.234 V | 3.366 V | TLV1117-3.3 DS, Table 6.6 | ### Crystal Oscillators A 12 MHz crystal with +/-20 ppm tolerance at 25 C: - 20 ppm of 12 MHz = 240 Hz | Net | Nominal | Min | Max | Source | |-----|---------|-----|-----|--------| | XTAL_OUT | 12 000 000 Hz | 11 999 760 Hz | 12 000 240 Hz | ABM8 DS, ppm spec | ### Logic Signals Use the component's VIH and VIL thresholds, not 50% of VCC. | Net | Type | Min | Max | Source | |-----|------|-----|-----|--------| | I2C_SDA (high) | V_OH | 2.97 V | 3.63 V | GPIO spec, 0.9*VCC to 1.1*VCC | | I2C_SCL (low) | V_OL | 0 V | 0.33 V | GPIO spec, V_OL max 0.1*VCC | ## Test Specification Document Structure Store the spec as a versioned table. One row per measurement. | ID | Test Point | Net | Phase Name | Measurement Type | Nominal | Min | Max | Unit | Datasheet Ref | |----|-----------|-----|-----------|-----------------|---------|-----|-----|------|---------------| | M01 | TP1 | VCC_3V3 | power_rails | DC voltage | 3.300 | 3.234 | 3.366 | V | TLV1117-3.3 DS Table 6.6 | | M02 | TP2 | VCC_5V0 | power_rails | DC voltage | 5.000 | 4.750 | 5.250 | V | USB 2.0 spec section 7.2.1 | | M03 | TP3 | XTAL_OUT | clock_check | Frequency | 12000000 | 11999760 | 12000240 | Hz | ABM8 DS ppm spec | | M04 | TP4 | I2C_SDA | i2c_bus | DC voltage | 3.300 | 2.970 | 3.630 | V | MCU GPIO spec | | M05 | TP5 | VBAT | battery_voltage | DC voltage | 3.700 | 3.000 | 4.200 | V | Li-ion cell spec | | M06 | TP6 | LED_PWM | led_driver | Duty cycle | 50.0 | 45.0 | 55.0 | % | Firmware requirement | Keep this table in a CSV or spreadsheet under version control alongside the firmware. Tag each spec revision to the hardware revision it applies to. ## Translating the Spec into OpenHTF Phases Each phase corresponds to a functional block in the spec. Measurements within a phase map to individual rows by ID. ```python filename="tests/pcba_functional_test.py" import openhtf as htf from openhtf.util import units from tofupilot.openhtf import TofuPilot from plugs.dmm import DmmPlug from plugs.counter import FrequencyCounterPlug @htf.plug(dmm=DmmPlug) @htf.measures( htf.Measurement("vcc_3v3_voltage") .in_range(minimum=3.234, maximum=3.366) .with_units(units.VOLT) .doc("M01: VCC_3V3 at TP1, TLV1117-3.3 DS Table 6.6"), htf.Measurement("vcc_5v0_voltage") .in_range(minimum=4.750, maximum=5.250) .with_units(units.VOLT) .doc("M02: VCC_5V0 at TP2, USB 2.0 spec 7.2.1"), ) def power_rails(test, dmm): test.measurements.vcc_3v3_voltage = dmm.measure_dc_voltage(channel=1) test.measurements.vcc_5v0_voltage = dmm.measure_dc_voltage(channel=2) @htf.plug(counter=FrequencyCounterPlug) @htf.measures( htf.Measurement("xtal_frequency") .in_range(minimum=11_999_760, maximum=12_000_240) .with_units(units.HERTZ) .doc("M03: XTAL_OUT at TP3, ABM8 DS ppm spec"), ) def clock_check(test, counter): test.measurements.xtal_frequency = counter.measure_frequency(channel=1) @htf.plug(dmm=DmmPlug) @htf.measures( htf.Measurement("i2c_sda_high_voltage") .in_range(minimum=2.970, maximum=3.630) .with_units(units.VOLT) .doc("M04: I2C_SDA at TP4, MCU GPIO spec"), ) def i2c_bus(test, dmm): test.measurements.i2c_sda_high_voltage = dmm.measure_dc_voltage(channel=3) @htf.plug(dmm=DmmPlug) @htf.measures( htf.Measurement("vbat_voltage") .in_range(minimum=3.000, maximum=4.200) .with_units(units.VOLT) .doc("M05: VBAT at TP5, Li-ion cell spec"), ) def battery_voltage(test, dmm): test.measurements.vbat_voltage = dmm.measure_dc_voltage(channel=4) @htf.plug(counter=FrequencyCounterPlug) @htf.measures( htf.Measurement("led_pwm_duty_cycle") .in_range(minimum=45.0, maximum=55.0) .doc("M06: LED_PWM at TP6, firmware requirement"), ) def led_driver(test, counter): test.measurements.led_pwm_duty_cycle = counter.measure_duty_cycle(channel=2) def main(): test = htf.Test( power_rails, clock_check, i2c_bus, battery_voltage, led_driver, test_name="PCBA Functional Test", ) with TofuPilot(test): test.execute(test_start=lambda: input("Enter DUT serial number: ")) if __name__ == "__main__": main() ``` The `.doc()` string on each measurement links it back to the spec ID, making the test output self-auditing. When a measurement fails in TofuPilot, you can trace it directly to the datasheet row that defined the limit. ## Tracking Spec Coverage in TofuPilot After your first run, cross-reference the spec table against the TofuPilot measurement list: | Spec ID | Measurement Name | TofuPilot Status | |---------|-----------------|-----------------| | M01 | vcc_3v3_voltage | present | | M02 | vcc_5v0_voltage | present | | M03 | xtal_frequency | present | | M04 | i2c_sda_high_voltage | present | | M05 | vbat_voltage | present | | M06 | led_pwm_duty_cycle | present | A measurement that never fails across hundreds of units may indicate limits that are too loose. Review the distribution in TofuPilot's measurement analytics and tighten limits if the histogram shows all values clustered far from the edges. ### Updating Limits When hardware revision B changes the LDO, update the spec table and code together: ```python filename="tests/pcba_functional_test.py" from openhtf.util import units # AP2112K-3.3: Vout accuracy +/-1.5%, load reg 0.3% # Min: 3.300 * (1 - 0.015 - 0.003) = 3.2406 # Max: 3.300 * (1 + 0.015 + 0.003) = 3.3594 htf.Measurement("vcc_3v3_voltage") .in_range(minimum=3.2406, maximum=3.3594) .with_units(units.VOLT) .doc("M01: VCC_3V3 at TP1, AP2112K-3.3 DS rev B"), ``` Commit the spec table update and the code change in the same commit so the audit trail stays intact. ### Motor and Actuator Testing with TofuPilot URL: https://www.tofupilot.com/guides/motor-and-actuator-testing-with-tofupilot Learn how to automate motor and actuator production tests, capture torque, speed, and vibration data, and track results with TofuPilot. # Motor and Actuator Testing with TofuPilot Every motor that leaves your line needs to spin at the right speed, draw the right current, and not vibrate itself apart. Manual spot checks don't scale. You need automated tests that capture torque curves, speed profiles, and vibration signatures for every unit. TofuPilot and OpenHTF let you build those tests in Python and track every result across production. ## What Motor Tests Cover Production motor testing typically includes: | Test Type | What It Measures | Why It Matters | |---|---|---| | No-load speed | RPM at rated voltage, no torque | Winding and magnet integrity | | Stall torque | Maximum torque at zero RPM | Mechanical strength | | Current draw | Amps at no-load, rated load, stall | Efficiency, winding shorts | | Back-EMF | Generated voltage when spun externally | Magnet and winding quality | | Vibration | Acceleration spectrum at operating speed | Bearing health, balance | | Insulation resistance | Megohm reading winding-to-case | Safety, dielectric integrity | ## Prerequisites - Python 3.8+ with `openhtf` and `tofupilot` installed - A dynamometer or torque sensor with serial/GPIB/USB interface - A programmable power supply for motor drive - Optional: accelerometer or vibration sensor ## Step 1: Set Up the No-Load Test The no-load test is your first gate. A motor that can't reach rated speed with no load has a fundamental problem. ```python filename="motor_noload_test.py" import openhtf as htf from openhtf.util import units import time @htf.measures( htf.Measurement("no_load_speed_rpm") .with_units(units.HERTZ) # RPM stored as value .in_range(2850, 3150) .doc("No-load speed at rated voltage"), htf.Measurement("no_load_current_a") .with_units(units.AMPERE) .at_most(0.5) .doc("No-load current draw"), htf.Measurement("startup_time_ms") .with_units(units.MILLISECOND) .at_most(200) .doc("Time to reach 90% rated speed"), ) def no_load_test(test, psu, tachometer): """Run motor at rated voltage with no load attached.""" psu.set_voltage(24.0) psu.output_on() start = time.time() # Wait for motor to reach steady state time.sleep(2.0) speed = tachometer.read_rpm() current = psu.measure_current() startup = tachometer.get_startup_time_ms() test.measurements.no_load_speed_rpm = speed test.measurements.no_load_current_a = current test.measurements.startup_time_ms = startup psu.output_off() ``` ## Step 2: Capture Torque-Speed Curves The torque-speed curve characterizes motor performance across its operating range. Capture it as a multi-dimensional measurement. ```python filename="motor_torque_curve.py" import openhtf as htf from openhtf.util import units import numpy as np @htf.measures( htf.Measurement("torque_speed_curve") .with_dimensions(units.HERTZ) # RPM as dimension .doc("Torque vs speed characteristic curve"), htf.Measurement("rated_torque_nm") .in_range(0.45, 0.55) .doc("Torque at rated speed"), htf.Measurement("efficiency_at_rated") .in_range(0.80, 1.0) .doc("Efficiency at rated operating point"), ) def torque_curve_test(test, psu, dyno): """Sweep load from no-load to stall, capture torque at each point.""" psu.set_voltage(24.0) psu.output_on() time.sleep(1.0) torque_points = [] speed_points = [] for load_pct in range(0, 105, 5): dyno.set_load_percent(load_pct) time.sleep(0.5) # Settle time torque = dyno.read_torque_nm() speed = dyno.read_rpm() torque_points.append(torque) speed_points.append(speed) test.measurements.torque_speed_curve[speed] = torque # Find rated operating point (closest to rated speed) rated_idx = np.argmin(np.abs(np.array(speed_points) - 3000)) rated_torque = torque_points[rated_idx] rated_speed = speed_points[rated_idx] test.measurements.rated_torque_nm = rated_torque # Calculate efficiency: P_mech / P_elec p_mech = rated_torque * rated_speed * 2 * np.pi / 60 p_elec = 24.0 * psu.measure_current() test.measurements.efficiency_at_rated = p_mech / p_elec dyno.set_load_percent(0) psu.output_off() ``` ## Step 3: Add Vibration Analysis Vibration testing catches bearing defects, rotor imbalance, and misalignment before they become field failures. ```python filename="motor_vibration_test.py" import openhtf as htf from openhtf.util import units import numpy as np @htf.measures( htf.Measurement("vibration_rms_g") .at_most(0.5) .doc("Overall vibration RMS in g"), htf.Measurement("vibration_spectrum") .with_dimensions(units.HERTZ) .doc("Vibration frequency spectrum"), htf.Measurement("dominant_frequency_hz") .with_units(units.HERTZ) .doc("Highest amplitude frequency component"), ) def vibration_test(test, psu, accelerometer): """Measure vibration at rated speed.""" psu.set_voltage(24.0) psu.output_on() time.sleep(3.0) # Wait for steady state # Capture 1 second of acceleration data at 10 kHz raw_data = accelerometer.capture(duration_s=1.0, sample_rate=10000) # RMS vibration rms = np.sqrt(np.mean(raw_data ** 2)) test.measurements.vibration_rms_g = rms # FFT for spectrum fft_vals = np.abs(np.fft.rfft(raw_data)) freqs = np.fft.rfftfreq(len(raw_data), d=1/10000) for freq, amplitude in zip(freqs[1:500], fft_vals[1:500]): test.measurements.vibration_spectrum[freq] = amplitude # Dominant frequency peak_idx = np.argmax(fft_vals[1:]) + 1 test.measurements.dominant_frequency_hz = freqs[peak_idx] psu.output_off() ``` ## Step 4: Build the Full Test Sequence Combine all motor tests into a single production sequence: ```python filename="motor_production_test.py" import openhtf as htf from tofupilot import TofuPilotClient def main(): test = htf.Test( no_load_test, torque_curve_test, vibration_test, ) test.add_output_callbacks( TofuPilotClient().as_openhtf_callback( procedure_id="motor-fct", procedure_name="Motor Production FCT", ) ) test.execute(test_start=htf.PhaseDescriptor.wrap( lambda test: setattr( test, "dut_id", input("Scan motor serial number: ") ) )) if __name__ == "__main__": main() ``` ## Step 5: Monitor Production Trends Once tests are running, use TofuPilot to catch drift before it causes failures: - **No-load speed trending down**: Possible demagnetization or increased friction - **Current draw creeping up**: Winding insulation degradation or bearing wear - **Vibration RMS increasing**: Bearing failure progression - **Efficiency dropping**: Look at both electrical and mechanical subsystems Set measurement limits with margin. If your spec is 2850 to 3150 RPM, consider tighter production limits of 2900 to 3100 to catch drift early. TofuPilot's control charts show these trends automatically. A shift in the mean or increasing standard deviation shows up before parts start failing. ## Actuator-Specific Considerations Linear actuators and servo motors need additional tests: | Actuator Type | Additional Tests | |---|---| | Linear actuator | Stroke length, extension force, retraction speed | | Servo motor | Position accuracy, settling time, step response | | Stepper motor | Holding torque, step accuracy, resonance points | | Solenoid | Pull-in voltage, drop-out voltage, response time | The test structure is the same. Define measurements with limits, capture the data, let TofuPilot track everything. ### Track Repair and Rework Data URL: https://www.tofupilot.com/guides/how-to-track-repair-and-rework-data-with-tofupilot Learn how to structure OpenHTF tests so repair loops, rework actions, and retests all link to the same serial number in TofuPilot. Production testing catches defects, but the real value comes from closing the loop: diagnosing failures, repairing units, and retesting them. TofuPilot links every retest to the original serial number so you get a complete history of each unit's journey through your repair process. ## The Repair Loop A typical repair workflow follows this cycle: a unit fails a test, a technician diagnoses the root cause, performs a repair, and the unit goes back through testing. Without structured data, this history gets lost in spreadsheets or paper logs. TofuPilot solves this by keying everything to the serial number. Every test run against the same DUT automatically appears in its unit history. You don't need special configuration. Just use the same serial number when retesting. ## Structure Your Initial Test Start with a standard OpenHTF test that measures your DUT and uploads results to TofuPilot. ```python filename="functional_test.py" import openhtf as htf from openhtf.util import units from tofupilot.openhtf import TofuPilot @htf.measures( htf.Measurement("output_voltage") .in_range(minimum=4.8, maximum=5.2) .with_units(units.VOLT), htf.Measurement("current_draw") .in_range(minimum=0.095, maximum=0.105) .with_units(units.AMPERE), ) def functional_check(test): test.measurements.output_voltage = 5.05 test.measurements.current_draw = 0.1012 def main(): test = htf.Test(functional_check) with TofuPilot(test): test.execute(test_start=lambda: "SN-20260312-001") if __name__ == "__main__": main() ``` When this test fails, the unit enters your repair queue. ## Record Repair Actions as Metadata After a technician diagnoses and repairs the unit, capture that context in the retest. Use phase-level metadata to record what was found and what was done. ```python filename="retest_after_repair.py" import openhtf as htf from openhtf.util import units from tofupilot.openhtf import TofuPilot @htf.measures( htf.Measurement("repair_code"), htf.Measurement("failure_category"), htf.Measurement("repair_action"), ) def record_repair_info(test): # These values come from your repair technician's input test.measurements.repair_code = "RC-042" test.measurements.failure_category = "solder_bridge" test.measurements.repair_action = "reworked_U3_solder_joints" @htf.measures( htf.Measurement("output_voltage") .in_range(minimum=4.8, maximum=5.2) .with_units(units.VOLT), htf.Measurement("current_draw") .in_range(minimum=0.095, maximum=0.105) .with_units(units.AMPERE), ) def functional_recheck(test): test.measurements.output_voltage = 5.01 test.measurements.current_draw = 0.1003 def main(): test = htf.Test(record_repair_info, functional_recheck) with TofuPilot(test): # Same serial number links this retest to the original failure test.execute(test_start=lambda: "SN-20260312-001") if __name__ == "__main__": main() ``` The `record_repair_info` phase stores the diagnosis and repair action alongside the retest measurements. This data shows up in TofuPilot's run detail view, tied to the same unit. ## Use Failure Categories Consistently Define a standard set of failure categories and repair codes across your team. Consistent naming lets you filter and aggregate repair data later. Common failure categories for PCBA testing: | Category | Description | |----------|-------------| | `solder_bridge` | Unintended solder connection between pads | | `cold_joint` | Insufficient solder wetting | | `missing_component` | Component not placed during assembly | | `wrong_value` | Incorrect component value populated | | `damaged_component` | Component damaged during handling or reflow | | `pcb_defect` | Board-level issue (trace crack, via failure) | Store these as string measurements so they're searchable in TofuPilot. ## Track Multiple Repair Cycles Some units need more than one repair attempt. Each retest creates a new run against the same serial number. TofuPilot's unit history view shows the full chain: initial fail, first repair attempt, second repair attempt, and final pass. ```python filename="second_repair_retest.py" import openhtf as htf from openhtf.util import units from tofupilot.openhtf import TofuPilot @htf.measures( htf.Measurement("repair_code"), htf.Measurement("failure_category"), htf.Measurement("repair_action"), htf.Measurement("repair_cycle"), ) def record_repair_info(test): test.measurements.repair_code = "RC-043" test.measurements.failure_category = "cold_joint" test.measurements.repair_action = "reflowed_C12_pads" test.measurements.repair_cycle = 2 @htf.measures( htf.Measurement("output_voltage") .in_range(minimum=4.8, maximum=5.2) .with_units(units.VOLT), htf.Measurement("current_draw") .in_range(minimum=0.095, maximum=0.105) .with_units(units.AMPERE), ) def functional_recheck(test): test.measurements.output_voltage = 4.98 test.measurements.current_draw = 0.0998 def main(): test = htf.Test(record_repair_info, functional_recheck) with TofuPilot(test): test.execute(test_start=lambda: "SN-20260312-001") if __name__ == "__main__": main() ``` Adding a `repair_cycle` measurement makes it easy to count how many attempts each unit needed. ## Analyze Repair Trends in TofuPilot Once your repair data flows into TofuPilot, the dashboard gives you what you need without writing analysis scripts. **Unit history** shows every test run for a serial number in chronological order. You can see exactly when a unit failed, what was repaired, and whether the retest passed. **Failure Pareto charts** rank your failure categories by frequency. If `solder_bridge` dominates, that's a signal to investigate your reflow profile or stencil design. **FPY trends** reflect your repair effectiveness over time. A rising FPY after process changes confirms the fix is working. A unit that keeps failing the same test after multiple repairs may point to a deeper design issue. ## Separate Test Procedures for Initial Test and Retest For traceability, consider using distinct procedure names for initial tests and retests. TofuPilot groups runs by procedure, so this separation makes reporting cleaner. ```python filename="retest_procedure.py" import openhtf as htf from openhtf.util import units from tofupilot.openhtf import TofuPilot @htf.measures( htf.Measurement("repair_code"), htf.Measurement("repair_action"), ) def record_repair_info(test): test.measurements.repair_code = "RC-042" test.measurements.repair_action = "reworked_U3_solder_joints" @htf.measures( htf.Measurement("output_voltage") .in_range(minimum=4.8, maximum=5.2) .with_units(units.VOLT), ) def functional_recheck(test): test.measurements.output_voltage = 5.02 def main(): test = htf.Test(record_repair_info, functional_recheck) with TofuPilot(test, procedure_name="PCBA Functional Test - Retest"): test.execute(test_start=lambda: "SN-20260312-001") if __name__ == "__main__": main() ``` This way you can compare FPY between initial tests and retests independently, while the unit history still ties everything together under one serial number. ### Design an End-of-Line Operator Screen URL: https://www.tofupilot.com/guides/how-to-design-an-end-of-line-test-operator-screen An EOL operator screen must be fast, clear, and error-proof. Learn how to design the interface for end-of-line test stations. # How to Design an End-of-Line Test Operator Screen An end-of-line (EOL) test operator runs the same test hundreds of times per shift. The screen they see must communicate three things instantly: what to do, whether it passed, and what to do next. This guide covers design principles for EOL operator screens and how to implement them with OpenHTF and TofuPilot. ## Design Principles ### 1. The 3-Second Rule An operator should understand the screen state within 3 seconds of looking at it. This means: | Element | Requirement | |---------|------------| | Pass/fail indicator | Full-screen green or red background, visible from 3 meters | | Current step | Large text showing what's happening now | | Operator action | Bold instruction when input is needed | | Everything else | Secondary, smaller, or hidden | ### 2. Minimize Decisions Every decision the operator makes is a chance for error. Reduce them: | Bad | Better | |-----|--------| | Type the serial number | Scan the barcode | | Choose the product variant | Auto-detect from barcode prefix | | Select the test procedure | One station, one procedure | | Click "Start Test" | Auto-start after barcode scan | | Click "Print Label" | Auto-print on pass | ### 3. Error-Proof the Workflow | Risk | Mitigation | |------|-----------| | Wrong serial format | Validate barcode format before starting | | Testing the same unit twice | Warn if serial was already tested today | | Skipping the test | Lock packaging station until test passes | | Missing a failure | Audible alert on fail, require acknowledgment | ## Prerequisites - Python 3.10+ - OpenHTF installed (`pip install openhtf`) - TofuPilot Python SDK installed (`pip install tofupilot`) ## Step 1: Design the Test Flow An EOL test should follow this sequence: ``` Scan barcode → Auto-start → Run phases → Show result → Ready for next unit ``` No menus, no configuration, no extra clicks. ```python filename="eol_operator.py" import openhtf as htf from openhtf.util import units from openhtf import PhaseResult @htf.measures( htf.Measurement("supply_current_mA") .in_range(minimum=90, maximum=110) .with_units(units.MILLIAMPERE), ) def phase_power_up(test): """Apply power and verify current draw.""" test.measurements.supply_current_mA = 101.2 @htf.measures( htf.Measurement("firmware_version").equals("3.1.0"), ) def phase_firmware(test): """Check firmware version matches production release.""" test.measurements.firmware_version = "3.1.0" @htf.measures( htf.Measurement("self_test").equals("PASS"), ) def phase_self_test(test): """Run the DUT's built-in self-test.""" test.measurements.self_test = "PASS" @htf.measures( htf.Measurement("output_voltage_V") .in_range(minimum=11.8, maximum=12.2) .with_units(units.VOLT), ) def phase_output_check(test): """Measure main output voltage.""" test.measurements.output_voltage_V = 12.03 ``` ## Step 2: Fail Fast Put the most likely failure first. If power-up fails, skip everything else. The operator sees the failure in under 5 seconds instead of waiting for the full test cycle. ```python filename="eol_operator.py" @htf.measures( htf.Measurement("power_good").equals("PASS"), ) def phase_power_good(test): """Quick power check before running full test.""" result = "PASS" test.measurements.power_good = result if result != "PASS": return PhaseResult.STOP ``` ## Step 3: Connect and Stream TofuPilot's operator UI handles the display. The operator sees each phase progress, measurements with limits, and a full-screen pass/fail result. ```python filename="eol_operator.py" from tofupilot.openhtf import TofuPilot test = htf.Test( phase_power_good, phase_power_up, phase_firmware, phase_self_test, phase_output_check, ) with TofuPilot(test): test.execute(test_start=lambda: input("Scan serial: ")) ``` ## Screen Layout Recommendations | Zone | Content | Size | |------|---------|------| | Top bar | Station name, current time, units tested today | Small | | Center | Current phase name and status, or pass/fail result | Large (60% of screen) | | Prompt area | Operator instruction when input is needed | Medium, highlighted | | Bottom | Last 5 results (pass/fail per serial) | Small, scrolling | ### Color Coding | State | Color | Meaning | |-------|-------|---------| | Idle | Gray or blue | Waiting for barcode scan | | Running | Blue | Test in progress | | Pass | Green | Unit passed all phases | | Fail | Red | Unit failed one or more phases | | Prompt | Yellow/amber | Waiting for operator input | ## Cycle Time Optimization for EOL EOL tests run continuously. Every second of cycle time matters. | Optimization | Effect | |-------------|--------| | Auto-start after scan | Saves 2-3 seconds per unit | | No unnecessary prompts | Saves 5-10 seconds per prompt | | Fail-fast phase ordering | Reduces average cycle time on bad units by 50%+ | | Parallel instrument commands | Saves 100-500ms per phase | | Kiosk mode (no browser chrome) | Prevents accidental navigation | ## What to Avoid | Mistake | Why | |---------|-----| | Small pass/fail text | Operator must lean in to read it | | Detailed measurement tables during test | Distracts from the current step | | Multiple windows or tabs | Operator gets lost | | Login screens | Operators share stations across shifts | | Dark theme in bright factory lighting | Low contrast, hard to read | | Animations or transitions | Slow down perception of state changes | ### Test Fixture Design and Management URL: https://www.tofupilot.com/guides/test-fixture-design-and-management-with-tofupilot Learn how to track test fixture health, calibration, and lifecycle using TofuPilot's station monitoring and measurement trending. # Test Fixture Design and Management with TofuPilot The test fixture is the most underappreciated part of hardware test infrastructure. A worn pogo pin or a loose cable causes more false failures than bad product. TofuPilot helps you monitor fixture health through measurement trends and station-specific analytics. ## What Makes a Test Fixture A test fixture connects the DUT (device under test) to the test instruments. It typically includes: | Component | Purpose | Failure mode | |-----------|---------|-------------| | Pogo pins / spring probes | Electrical contact to DUT | Wear, contamination, broken springs | | Alignment hardware | Position DUT consistently | Wear, loosening over time | | Cable harness | Connect fixture to instruments | Broken wires, loose connectors | | Pneumatic actuator | Clamp DUT against probes | Seal failure, pressure loss | | Guide pins | Align PCB to probe positions | Wear, bending | ## Monitoring Fixture Health Through Test Data You don't need dedicated fixture tests. Your production test data already contains fixture health signals. ### Contact Resistance as a Health Indicator If your test includes a continuity or resistance measurement, track it per station. ```python filename="fixture_health.py" from tofupilot import TofuPilotClient client = TofuPilotClient() # This measurement doubles as a DUT test AND a fixture health indicator client.create_run( procedure_id="BOARD-FUNCTIONAL", unit_under_test={"serial_number": serial}, run_passed=True, steps=[{ "name": "Contact Verification", "step_type": "measurement", "status": True, "measurements": [ {"name": "probe_contact_mohm", "value": 35, "unit": "mohm", "limit_high": 100}, ], }], ) ``` Track `probe_contact_mohm` per station over time: | Week | Station 1 | Station 2 | Station 3 | |------|-----------|-----------|-----------| | 1 | 25 mohm | 28 mohm | 30 mohm | | 4 | 28 mohm | 30 mohm | 45 mohm | | 8 | 30 mohm | 32 mohm | 72 mohm | | 12 | 32 mohm | 35 mohm | 95 mohm | Station 3's contact resistance is climbing fast. Service the pogo pins before it hits the 100 mohm limit and starts causing false failures. ### Measurement Variance as a Health Indicator Even when measurements pass, increasing variance signals fixture problems. A voltage measurement that bounces between 3.29V and 3.33V has more variance than one that reads 3.31V consistently. High variance on a specific station means something is intermittent: a loose wire, a dirty probe, or a worn alignment pin. ### Cycle Time as a Health Indicator If a test normally takes 42 seconds and starts taking 55 seconds on one station, something changed: - Pneumatic actuator slowing down (low air pressure, seal wear) - Instrument communication timeout (loose cable, aging connector) - Retry loops from intermittent contacts ## Fixture Lifecycle Tracking ### Maintenance Schedule Based on Data Replace calendar-based maintenance with data-driven maintenance. | Approach | Schedule | Basis | |----------|----------|-------| | Calendar-based | Every 3 months | Time elapsed | | Count-based | Every 50,000 cycles | Insertion count | | Condition-based | When data shows degradation | Measurement trends | Condition-based maintenance costs less and catches problems earlier. TofuPilot's measurement trends tell you which fixtures need attention now and which are fine. ### Tracking Pogo Pin Life Pogo pins have a rated cycle life (typically 100,000-1,000,000 cycles). Track cycles per fixture. ```python filename="fixture_tracking.py" # Track fixture cycle count alongside DUT test client.create_run( procedure_id="BOARD-FUNCTIONAL", unit_under_test={"serial_number": serial}, run_passed=True, steps=[{ "name": "Contact Verification", "step_type": "measurement", "status": True, "measurements": [ {"name": "probe_contact_mohm", "value": contact_r, "unit": "mohm", "limit_high": 100}, {"name": "fixture_cycle_count", "value": get_fixture_count(), "unit": "cycles"}, ], }], ) ``` When contact resistance starts rising, check the cycle count. If you're at 80% of rated life, schedule replacement during the next planned downtime. ## Fixture Design Best Practices | Practice | Why | |----------|-----| | Use keyed connectors | Prevent incorrect cable connections | | Include a "golden board" test point | Verify fixture health independently from DUT | | Minimize cable length | Reduces noise, improves measurement accuracy | | Use separate ground pins | Don't rely on one ground contact for the whole fixture | | Design for probe replacement | Make pogo pins individually replaceable | | Add alignment features | Ensure consistent DUT positioning | | Include interlock sensors | Detect when DUT is properly seated | ### Golden Board Verification A "golden board" is a known-good reference board. Run it through the fixture periodically to verify the fixture is working correctly. ```python filename="golden_board_check.py" # Weekly golden board verification GOLDEN_SERIAL = "GOLDEN-001" EXPECTED_VCC = 3.310 # Known value from initial characterization vcc = measure_voltage() drift = abs(vcc - EXPECTED_VCC) client.create_run( procedure_id="FIXTURE-GOLDEN-BOARD-CHECK", unit_under_test={"serial_number": GOLDEN_SERIAL}, run_passed=drift < 0.010, # Less than 10mV drift steps=[{ "name": "Golden Board Verification", "step_type": "measurement", "status": drift < 0.010, "measurements": [ {"name": "vcc_3v3", "value": vcc, "unit": "V", "limit_low": 3.300, "limit_high": 3.320}, {"name": "drift_from_reference_mv", "value": drift * 1000, "unit": "mV", "limit_high": 10}, ], }], ) ``` If the golden board measurement drifts, the fixture or instrument has changed, not the board. ## Common Fixture Failures and Detection | Failure | Detection in TofuPilot | Fix | |---------|----------------------|-----| | Worn pogo pins | Rising contact resistance trend | Replace affected pins | | Loose cable | Intermittent measurement failures (random, no pattern) | Reseat or replace cable | | Alignment wear | Increasing measurement variance across all channels | Replace guide pins | | Contamination | Gradual measurement shift, contact resistance increase | Clean probes and PCB contacts | | Pneumatic leak | Longer cycle times, intermittent contacts | Replace seals | ## Multi-Fixture Management For production lines with multiple fixtures for the same test: 1. Give each fixture a unique identifier 2. Track which fixture ran each test (station ID in TofuPilot) 3. Compare performance across fixtures 4. Rotate golden board checks across all fixtures If Fixture A has 98% yield and Fixture B has 94% yield on the same product, Fixture B needs attention. TofuPilot's station comparison view shows this at a glance. ### Sensor Calibration at Scale with TofuPilot URL: https://www.tofupilot.com/guides/sensor-calibration-at-scale-with-tofupilot Learn how to automate sensor calibration workflows, track calibration certificates, and manage multi-point calibration data with TofuPilot. # Sensor Calibration at Scale with TofuPilot Calibrating 10 sensors is a manual job. Calibrating 10,000 requires a system. You need multi-point reference measurements, pass/fail against tolerance bands, calibration certificates, and traceability back to reference standards. TofuPilot handles the data side so you can focus on the calibration procedure itself. ## What Sensor Calibration Involves Production sensor calibration typically follows this flow: 1. Apply known reference stimuli (temperature, pressure, force, etc.) 2. Read the sensor output at each reference point 3. Calculate error, linearity, and hysteresis 4. Apply correction factors if needed 5. Verify corrected output meets specifications 6. Generate a calibration certificate ## Prerequisites - Python 3.8+ with `openhtf` and `tofupilot` installed - A reference standard (calibrated source or reference sensor) - A data acquisition system or instrument to read sensor output ## Step 1: Define Multi-Point Calibration Measurements A typical calibration uses 5 to 11 reference points across the sensor's range. Define measurements for each point: ```python filename="sensor_cal.py" import openhtf as htf from openhtf.util import units CAL_POINTS = [0, 25, 50, 75, 100] # Percent of full scale measures = [] for pct in CAL_POINTS: measures.append( htf.Measurement(f"error_at_{pct}pct") .with_units(units.PERCENT) .in_range(-0.5, 0.5) .doc(f"Measurement error at {pct}% of full scale") ) measures.append( htf.Measurement("max_linearity_error") .with_units(units.PERCENT) .at_most(0.25) .doc("Maximum linearity deviation from best-fit line") ) measures.append( htf.Measurement("hysteresis") .with_units(units.PERCENT) .at_most(0.1) .doc("Maximum hysteresis between up and down sweep") ) @htf.measures(*measures) def calibration_test(test, reference_source, sensor_reader): """Run multi-point calibration with up and down sweep.""" full_scale = 100.0 # Adjust for your sensor range up_readings = {} down_readings = {} # Up sweep for pct in CAL_POINTS: ref_value = full_scale * pct / 100.0 reference_source.set_output(ref_value) time.sleep(2.0) # Settle time reading = sensor_reader.read() up_readings[pct] = reading error = (reading - ref_value) / full_scale * 100 setattr(test.measurements, f"error_at_{pct}pct", error) # Down sweep for hysteresis for pct in reversed(CAL_POINTS): ref_value = full_scale * pct / 100.0 reference_source.set_output(ref_value) time.sleep(2.0) reading = sensor_reader.read() down_readings[pct] = reading # Hysteresis: max difference between up and down readings max_hyst = max( abs(up_readings[p] - down_readings[p]) / full_scale * 100 for p in CAL_POINTS ) test.measurements.hysteresis = max_hyst # Linearity: deviation from best-fit line import numpy as np ref_vals = [full_scale * p / 100.0 for p in CAL_POINTS] read_vals = [up_readings[p] for p in CAL_POINTS] coeffs = np.polyfit(ref_vals, read_vals, 1) fit_vals = np.polyval(coeffs, ref_vals) linearity_errors = [(r - f) / full_scale * 100 for r, f in zip(read_vals, fit_vals)] test.measurements.max_linearity_error = max(abs(e) for e in linearity_errors) ``` ## Step 2: Store Calibration Curves as Multi-Dimensional Data Capture the full calibration curve, not just pass/fail: ```python filename="cal_curve_capture.py" import openhtf as htf from openhtf.util import units @htf.measures( htf.Measurement("calibration_curve") .with_dimensions(units.PERCENT) .doc("Full calibration curve: reference input vs sensor output"), htf.Measurement("correction_curve") .with_dimensions(units.PERCENT) .doc("Correction factors at each calibration point"), ) def capture_calibration_curve(test, reference_source, sensor_reader): """Capture dense calibration curve for post-processing.""" full_scale = 100.0 for pct in range(0, 101, 5): # 5% increments ref_value = full_scale * pct / 100.0 reference_source.set_output(ref_value) time.sleep(1.0) reading = sensor_reader.read() test.measurements.calibration_curve[pct] = reading # Correction factor: what to add to get the true value correction = ref_value - reading test.measurements.correction_curve[pct] = correction ``` TofuPilot stores the full curve. You can compare curves across sensors, detect batch variations, and track calibration drift over time. ## Step 3: Track Reference Standard Traceability Every calibration is only as good as its reference. Track which reference standard was used: ```python filename="cal_traceability.py" from tofupilot import TofuPilotClient client = TofuPilotClient() result = client.create_run( procedure_id="pressure-sensor-cal", unit_under_test={ "serial_number": sensor_sn, "part_number": "PS-500-A", }, run_passed=True, properties={ "reference_standard": "FLUKE-8845A-SN12345", "reference_cal_date": "2026-01-15", "reference_cal_due": "2027-01-15", "reference_cert_number": "CAL-2026-0042", "ambient_temp_c": 23.1, "ambient_humidity_pct": 45, "operator": "OP-003", }, ) ``` When an auditor asks about your traceability chain, every calibration run links to the reference standard, its own calibration certificate, and the environmental conditions during the procedure. ## Step 4: Manage Recalibration Schedules Sensors drift. Track when each unit was last calibrated and when it's due: ```python filename="recal_tracking.py" from tofupilot import TofuPilotClient from datetime import date, timedelta client = TofuPilotClient() # Get all calibration runs for a sensor type cal_runs = client.get_runs( procedure_id="pressure-sensor-cal", limit=5000, ) # Find latest calibration per serial number latest_cal = {} for run in cal_runs: sn = run.unit.serial_number if sn not in latest_cal or run.started_at > latest_cal[sn].started_at: latest_cal[sn] = run # Check for overdue recalibrations (12-month interval) cal_interval = timedelta(days=365) today = date.today() overdue = [] upcoming = [] for sn, run in latest_cal.items(): cal_date = run.started_at.date() due_date = cal_date + cal_interval if due_date < today: overdue.append((sn, due_date)) elif due_date < today + timedelta(days=30): upcoming.append((sn, due_date)) print(f"Overdue: {len(overdue)} sensors") print(f"Due within 30 days: {len(upcoming)} sensors") ``` ## Step 5: Detect Calibration Drift Compare calibration results over time to catch sensors that are drifting toward their tolerance limits: ```python filename="drift_detection.py" from tofupilot import TofuPilotClient client = TofuPilotClient() # Get calibration history for a specific sensor cal_history = client.get_runs( procedure_id="pressure-sensor-cal", unit_serial_number="PS-500-A-0042", limit=10, ) print(f"Calibration history for PS-500-A-0042:") print(f"{'Date':<12} {'Error@50%':<12} {'Linearity':<12} {'Status'}") print("-" * 48) for run in reversed(cal_history): error_50 = run.measurements.get("error_at_50pct", {}).get("value", "N/A") linearity = run.measurements.get("max_linearity_error", {}).get("value", "N/A") status = "PASS" if run.passed else "FAIL" print(f"{run.started_at.date()!s:<12} {error_50:<12} {linearity:<12} {status}") ``` If the error at 50% is 0.1% this year and was 0.05% last year, you know which direction it's heading. Replace or adjust before it fails. ### How to Build a PPAP Test Package with TofuPilot URL: https://www.tofupilot.com/guides/how-to-build-a-ppap-test-package-with-tofupilot Learn how to assemble PPAP test documentation from production test data using TofuPilot, including control plans, MSA, and process capability reports. # How to Build a PPAP Test Package with TofuPilot Your customer sent a PPAP request. You need control plans, Cpk reports, MSA studies, and dimensional data, all tied to specific part numbers and production runs. Most teams spend days pulling this together from spreadsheets and disconnected systems. TofuPilot captures the test data you need for PPAP submissions as part of your normal production workflow. No extra data entry required. ## What Is PPAP Production Part Approval Process (PPAP) is a standardized framework, defined in AIAG's PPAP manual, that proves your manufacturing process consistently produces parts meeting customer specifications. It's required across automotive (IATF 16949), and increasingly adopted in aerospace, medical devices, and industrial electronics. A PPAP submission typically includes 18 elements. Several depend directly on test data: | PPAP Element | What It Needs | TofuPilot Source | |---|---|---| | Control Plan | Test steps, limits, frequencies | Test procedure definitions | | Process Capability Study | Cpk/Ppk for critical dimensions | Measurement history | | MSA / Gage R&R | Measurement system analysis | Repeated measurement data | | Dimensional Results | Measured vs. spec for each characteristic | Run measurements with limits | | Material/Performance Test Results | Functional test pass/fail with data | Test run results | ## Prerequisites - A TofuPilot account with production test data - Python 3.8+ with the `tofupilot` client installed - At least 30 production runs for process capability calculations ## Step 1: Structure Your Tests Around Critical Characteristics PPAP reviewers care about Critical-to-Quality (CTQ) characteristics. Map each CTQ to a measurement in your test procedure. ```python filename="ppap_test.py" import openhtf as htf from openhtf.util import units # Map each CTQ characteristic to a measurement with spec limits @htf.measures( htf.Measurement("output_voltage_5v") .with_units(units.VOLT) .in_range(4.95, 5.05) .doc("CTQ-001: 5V rail accuracy"), htf.Measurement("current_draw_idle") .with_units(units.AMPERE) .in_range(0.0, 0.150) .doc("CTQ-002: Idle current consumption"), htf.Measurement("rise_time_ms") .with_units(units.MILLISECOND) .at_most(5.0) .doc("CTQ-003: Power-on rise time"), ) def power_rail_test(test): # Your instrument control code here test.measurements.output_voltage_5v = 5.01 test.measurements.current_draw_idle = 0.098 test.measurements.rise_time_ms = 3.2 ``` Use the `.doc()` field to store the CTQ identifier. This makes it easy to trace from PPAP documentation back to actual test results. ## Step 2: Collect Process Capability Data PPAP Level 3 submissions require process capability studies. You need at least 30 consecutive parts (AIAG recommends 300 for Ppk). TofuPilot calculates Cp and Cpk automatically from your measurement data. To pull capability data for your PPAP package: ```python filename="ppap_capability.py" from tofupilot import TofuPilotClient client = TofuPilotClient() # Get runs for a specific part number and procedure runs = client.get_runs( procedure_id="power-board-fct", unit_part_number="PWR-200-R3", limit=300, ) # Extract measurement values for each CTQ voltage_values = [ r.measurements["output_voltage_5v"].value for r in runs if r.measurements.get("output_voltage_5v") ] print(f"Sample size: {len(voltage_values)}") print(f"Mean: {sum(voltage_values) / len(voltage_values):.4f}") ``` TofuPilot's dashboard shows Cpk directly on measurement charts. Screenshot those for your submission, or export the raw data. ## Step 3: Run Gage R&R Studies Measurement System Analysis (MSA) proves your test equipment is capable. A typical Gage R&R study uses 10 parts, 3 operators, 3 trials each. ```python filename="gage_rr_study.py" import openhtf as htf from tofupilot import TofuPilotClient client = TofuPilotClient() # Tag each run with operator and trial info for MSA for operator in ["OP-A", "OP-B", "OP-C"]: for trial in range(1, 4): for part_sn in part_serial_numbers: # Run the test result = run_test(part_sn) # Upload with metadata for MSA grouping client.create_run( procedure_id="gage-rr-voltage", unit_under_test={ "serial_number": part_sn, "part_number": "PWR-200-R3", }, run_passed=result.passed, measurements=result.measurements, properties={ "operator": operator, "trial": trial, "study": "MSA-2026-Q1", }, ) ``` Filter by the `study` property in TofuPilot to pull all Gage R&R data for analysis. ## Step 4: Generate the Control Plan Your control plan documents what you test, how you test it, and what limits you use. Pull this directly from your test definitions: ```python filename="control_plan_export.py" from tofupilot import TofuPilotClient client = TofuPilotClient() # Get procedure definition with all measurements procedure = client.get_procedure("power-board-fct") print("| CTQ | Measurement | Lower Limit | Upper Limit | Unit | Method |") print("|-----|-------------|-------------|-------------|------|--------|") for m in procedure.measurements: print( f"| {m.doc or '-'} | {m.name} | " f"{m.lower_limit or '-'} | {m.upper_limit or '-'} | " f"{m.unit or '-'} | Automated FCT |" ) ``` This table goes directly into your PPAP control plan. When limits change, regenerate it from the same source of truth. ## Step 5: Package Dimensional and Performance Results PPAP requires actual measured values for a sample of parts (typically 5 from a significant production run). ```python filename="ppap_sample_results.py" from tofupilot import TofuPilotClient client = TofuPilotClient() # Get the first 5 passing runs from the initial production run sample_runs = client.get_runs( procedure_id="power-board-fct", unit_part_number="PWR-200-R3", passed=True, limit=5, ) for run in sample_runs: print(f"SN: {run.unit.serial_number}") for name, m in run.measurements.items(): status = "PASS" if m.passed else "FAIL" print(f" {name}: {m.value} {m.unit or ''} [{status}]") ``` ## Maintaining Your PPAP Package PPAP isn't a one-time exercise. When you change your process, you may need to resubmit. Track what triggers a PPAP update: - **Engineering change**: New part revision, updated test limits - **Process change**: New test equipment, different station configuration - **Supplier change**: New component source affecting test characteristics Because TofuPilot versions your test procedures and tracks station configurations, you can always trace which PPAP submission maps to which test setup. When a customer asks "what changed since your last submission," the answer is in your run history. ### What Is Test Observability URL: https://www.tofupilot.com/guides/what-is-test-observability Test observability gives full visibility into what your test systems are doing, why they fail, and how they perform. Learn how it applies to manufacturing test. # What Is Test Observability Test observability is the ability to understand what your test systems are doing, why units are failing, and how station performance is changing, all from the data the systems produce. It borrows from software engineering's observability concept (logs, metrics, traces) and applies it to manufacturing test. This guide covers what test observability means, how it differs from test data management, and what it looks like in practice. ## Observability vs Data Management | Aspect | Test Data Management | Test Observability | |--------|---------------------|-------------------| | Focus | Storing and retrieving test records | Understanding system behavior in real time | | Question answered | "What happened to this unit?" | "Why is yield dropping right now?" | | Data use | Compliance, traceability, audits | Debugging, optimization, early warning | | Timeliness | After the fact (batch reports) | Real-time (streaming dashboards) | | Scope | Individual test results | System-wide patterns and correlations | Test data management answers: did this unit pass? Test observability answers: why are 15% of units failing phase_voltage_check on station 3 since Tuesday? ## The Three Pillars of Test Observability Borrowing from software observability, test observability has three pillars: ### 1. Measurements (Metrics) Structured measurement data with units, limits, and timestamps. This is the equivalent of application metrics in software. | Metric | What It Shows | |--------|-------------| | First pass yield per station | Station health | | Measurement distribution per phase | Process stability | | Cycle time per unit | Throughput efficiency | | Failure rate per test step | Where defects concentrate | | Marginal rate per measurement | Early warning of drift | ### 2. Test Records (Logs) Detailed records of every test run: serial number, phases executed, measurements collected, pass/fail decisions, timestamps, and errors. | Record Field | Why It Matters | |-------------|---------------| | Serial number | Links test data to the physical unit | | Phase sequence | Shows exactly what ran and in what order | | Measurement values | The actual data for analysis | | Error messages | What went wrong when a phase failed | | Station ID | Which equipment ran the test | | Firmware/software version | Which test script version was used | ### 3. Traces (Correlation) The ability to trace a unit's journey across test stages: IQC, assembly, FCT, EOL, ORT. Traces connect upstream events to downstream outcomes. | Trace | What It Reveals | |-------|----------------| | Unit history | Every test this serial number has ever been through | | Lot traceability | All units from the same component lot | | Station correlation | Whether failures cluster on specific stations | | Temporal correlation | Whether failures cluster at specific times | ## What Observable Test Systems Look Like ### Without Observability The test engineer gets a call: "yield dropped on line 2." They walk to the station, review logs, export data to Excel, build charts, and three hours later identify the root cause. ### With Observability The test engineer sees an alert: "yield on station 3 dropped below 90% in the last hour. Top failure: phase_voltage_check. Measurement distribution shifted 200mV compared to yesterday." They open the dashboard, see the shift, correlate it with a supplier lot change at IQC, and contain the issue in minutes. ## Prerequisites - Python 3.10+ - OpenHTF installed (`pip install openhtf`) - TofuPilot Python SDK installed (`pip install tofupilot`) ## Step 1: Instrument Your Tests Every test phase should produce structured measurements. This is the foundation of observability. ```python filename="observable_test.py" import openhtf as htf from openhtf.util import units @htf.measures( htf.Measurement("supply_voltage_V") .in_range(minimum=4.9, maximum=5.1) .with_units(units.VOLT), htf.Measurement("supply_current_mA") .in_range(minimum=90, maximum=110) .with_units(units.MILLIAMPERE), ) def phase_power_check(test): """Measure and validate power supply characteristics.""" test.measurements.supply_voltage_V = 5.01 test.measurements.supply_current_mA = 98.5 @htf.measures( htf.Measurement("signal_amplitude_V") .in_range(minimum=1.8, maximum=2.2) .with_units(units.VOLT), ) def phase_signal_check(test): """Measure output signal amplitude.""" test.measurements.signal_amplitude_V = 2.01 ``` ## Step 2: Stream to TofuPilot TofuPilot provides the observability layer. Every run streams in real time with full measurement detail. ```python filename="observable_test.py" from tofupilot.openhtf import TofuPilot test = htf.Test( phase_power_check, phase_signal_check, ) with TofuPilot(test): test.execute(test_start=lambda: input("Scan serial: ")) ``` ## Step 3: Monitor and Respond TofuPilot provides the observability tools: | Tool | What It Shows | |------|-------------| | Yield dashboard | Real-time FPY across stations and procedures | | Failure Pareto | Which test steps fail most, ranked by frequency | | Measurement distributions | Histograms with limit overlays for every measurement | | Control charts | SPC charts detecting out-of-control conditions | | Station comparison | Side-by-side performance across stations | | Unit history | Full test trace for any serial number | | Alerts | Notifications when yield drops below threshold | ## Observability Maturity Levels | Level | Capability | Question Answered | |-------|-----------|------------------| | 1. Logging | Test results are stored | "Did this unit pass?" | | 2. Monitoring | Dashboards show yield and throughput | "How is the line performing?" | | 3. Alerting | Notifications on yield drops or measurement drift | "Is something going wrong right now?" | | 4. Debugging | Drill down into failures, correlate across stations and lots | "Why is this happening?" | | 5. Predicting | Use patterns to forecast future failures | "What will happen next?" | Most manufacturing operations are at level 1 or 2. Levels 3-4 are where test observability delivers the highest ROI. Level 5 is where predictive quality begins. ### Predictive Maintenance from Test Data URL: https://www.tofupilot.com/guides/predictive-maintenance-from-test-data-in-tofupilot Learn how to use production test data trends in TofuPilot to predict equipment failures and schedule maintenance before downtime occurs. # Predictive Maintenance from Test Data in TofuPilot Your test data already contains the signals for predictive maintenance. A test fixture that's wearing out doesn't fail suddenly. Its measurements drift first. TofuPilot's trending tools let you see the drift before it causes failures. ## What Test Data Reveals About Equipment Health Every test run is a health check on two things: the unit under test and the test equipment itself. When a measurement starts drifting, it could be the product changing or the test station changing. Separating the two is the key to predictive maintenance. | Signal | Product issue | Equipment issue | |--------|--------------|-----------------| | Measurement drift | Component lot variation | Fixture wear, calibration drift | | Increased variance | Mixed component populations | Loose connections, intermittent contacts | | Step change | Design revision, process change | Equipment swap, recalibration | | Station-specific failures | N/A (affects all stations) | Worn probes, faulty instruments | The distinguishing factor: if the drift or failure is specific to one station, it's the equipment. ## Using TofuPilot for Fixture Health Monitoring ### Track Key Measurements by Station Filter measurements by station to isolate equipment-related trends from product-related trends. ```python filename="fixture_health_check.py" from tofupilot import TofuPilotClient client = TofuPilotClient() # Get recent runs from a specific station runs = client.get_runs( procedure_id="BOARD-FUNCTIONAL", limit=500, ) # Track contact resistance trend (proxy for pogo pin wear) contact_readings = [] for run in runs: for step in run.get("steps", []): for m in step.get("measurements", []): if m["name"] == "contact_resistance_mohm": contact_readings.append({ "date": run["created_at"], "value": m["value"], "station": run.get("station_id"), }) # Rising contact resistance = pogo pins wearing out ``` ### Set Maintenance Thresholds Don't wait for a fixture to fail. Set maintenance thresholds based on measurement trends. | Measurement | Normal range | Maintenance threshold | Failure threshold | |------------|-------------|----------------------|-------------------| | Contact resistance | 10-50 mohm | > 80 mohm | > 150 mohm | | Probe alignment (via measurement variance) | < 0.5% RSD | > 1% RSD | > 2% RSD | | Cycle time | 40-45 s | > 55 s | > 70 s | When a station's contact resistance trend crosses the maintenance threshold, schedule pin replacement. Don't wait for it to hit the failure threshold. ### Monitor Cycle Time as a Health Indicator Test cycle time is an underused equipment health signal. A test that normally takes 42 seconds and starts taking 55 seconds is telling you something: - Instrument communication delays (aging GPIB cables, USB hub issues) - Actuator slowdown (pneumatic fixture, motorized probe) - Retry loops (intermittent contacts causing measurement retries) - Software issues (memory leaks in long-running test software) TofuPilot tracks cycle time for every run. A sudden increase is an early warning. ## Calibration Drift Detection Test instruments drift over time. A DMM that reads 3.300V today might read 3.305V next month. If your spec limit is 3.35V, that 5mV drift eats into your margin. TofuPilot helps detect calibration drift by comparing measurement distributions over time: 1. Establish a baseline distribution during the first month after calibration 2. Monitor the distribution monthly 3. If the mean shifts by more than 10% of the spec range, recalibrate This approach lets you move from calendar-based calibration (every 6 months regardless) to condition-based calibration (when the data shows drift). You save money on unnecessary calibrations and avoid the risk of running with a drifted instrument. ## From Reactive to Predictive | Approach | When you fix it | Cost | |----------|----------------|------| | Reactive | After it breaks | Highest: downtime + scrap + emergency repair | | Preventive | On a schedule | Medium: some unnecessary maintenance | | Predictive | When data shows early signs | Lowest: targeted maintenance, zero downtime | The data for predictive maintenance is already flowing through your test stations. TofuPilot stores it, trends it, and surfaces the early warning signs. You just need to look. ## Building a Maintenance Dashboard Create a monitoring view focused on equipment health: 1. **Station FPY comparison**: Drop in one station's yield = equipment issue 2. **Contact resistance trend per station**: Rising trend = fixture wear 3. **Cycle time trend per station**: Increasing trend = equipment slowdown 4. **Measurement variance per station**: Increasing variance = intermittent contacts Review this dashboard weekly. Most equipment issues give you days or weeks of warning before they cause production impact. The data is there. TofuPilot makes it visible. ### Add Measurements with Limits URL: https://www.tofupilot.com/guides/how-to-add-measurements-with-limits-in-tofupilot A complete reference for OpenHTF measurement types, validators, and units, with code examples for numeric ranges, exact matches, percentages, and marginal. Every measurement in a manufacturing test needs three things: a name, a value, and limits that define pass or fail. OpenHTF provides built-in validators for numeric ranges, exact matches, percentages, and regex patterns. When you log these measurements through TofuPilot, you get FPY, Cpk, and control charts automatically. This guide covers every measurement type with working code. ## Measurement Validators OpenHTF has six built-in validators: | Validator | Use Case | Example | |-----------|----------|---------| | `.in_range(min, max)` | Value within bounds | Voltage, current, resistance | | `.in_range(minimum=x)` | Value above minimum | Signal strength, gain | | `.in_range(maximum=x)` | Value below maximum | Latency, noise floor | | `.equals(value)` | Exact match | Firmware version, boolean flags | | `.within_percent(target, pct)` | Value within percentage of target | Crystal frequency, calibrated sensors | | `.matches_regex(pattern)` | String matches regex | MAC address, serial number format | ## Numeric Range: `.in_range()` The most common validator. Pass if the value falls between lower and upper limits. ```python filename="measurements/numeric_range.py" import openhtf as htf from openhtf.util import units @htf.measures( htf.Measurement("rail_3v3") .in_range(3.2, 3.4) .with_units(units.VOLT) .doc("3.3V supply rail voltage"), htf.Measurement("rail_5v0") .in_range(4.8, 5.2) .with_units(units.VOLT) .doc("5.0V supply rail voltage"), htf.Measurement("rail_1v8") .in_range(1.7, 1.9) .with_units(units.VOLT) .doc("1.8V core rail voltage"), htf.Measurement("idle_current") .in_range(0.05, 0.25) .with_units(units.AMPERE) .doc("Board idle current draw"), ) def test_power_rails(test): test.measurements.rail_3v3 = 3.31 test.measurements.rail_5v0 = 5.01 test.measurements.rail_1v8 = 1.82 test.measurements.idle_current = 0.12 ``` ### One-Sided Limits Use keyword arguments for minimum-only or maximum-only limits. ```python filename="measurements/one_sided.py" import openhtf as htf # Minimum only: pass if value >= -80 @htf.measures( htf.Measurement("signal_strength") .in_range(minimum=-80) .doc("Wi-Fi signal strength in dBm"), ) def test_signal(test): test.measurements.signal_strength = -65 # Maximum only: pass if value <= 100 @htf.measures( htf.Measurement("response_time") .in_range(maximum=100) .doc("Response latency in milliseconds"), ) def test_latency(test): test.measurements.response_time = 42 ``` ## Exact Match: `.equals()` Pass if the value exactly matches. Works with strings, booleans, and numbers. ```python filename="measurements/exact_match.py" import openhtf as htf # String match @htf.measures( htf.Measurement("firmware_version") .equals("2.1.0") .doc("Expected firmware version string"), ) def test_firmware(test): test.measurements.firmware_version = "2.1.0" # Boolean match @htf.measures( htf.Measurement("led_on") .equals(True) .doc("Power LED is illuminated"), htf.Measurement("error_flag") .equals(False) .doc("No error flags set"), ) def test_indicators(test): test.measurements.led_on = True test.measurements.error_flag = False ``` ## Percentage Tolerance: `.within_percent()` Pass if the value is within a percentage of a target. Useful for calibrated components. ```python filename="measurements/percentage.py" import openhtf as htf from openhtf.util import units @htf.measures( # 8MHz crystal, 1% tolerance -> 7.92MHz to 8.08MHz htf.Measurement("clock_freq") .within_percent(8_000_000, 1.0) .with_units(units.HERTZ) .doc("8MHz crystal oscillator frequency"), # 10kOhm resistor, 5% tolerance -> 9.5kOhm to 10.5kOhm htf.Measurement("pullup_resistance") .within_percent(10_000, 5.0) .with_units(units.OHM) .doc("I2C pullup resistor value"), ) def test_components(test): test.measurements.clock_freq = 8_000_100 test.measurements.pullup_resistance = 9_850 ``` `.within_percent(target, percent)` is equivalent to `.in_range(target * (1 - pct/100), target * (1 + pct/100))` but clearer in intent. ## Regex Match: `.matches_regex()` Pass if the string matches a regular expression. Useful for format validation. ```python filename="measurements/regex.py" import openhtf as htf @htf.measures( # MAC address format: AA:BB:CC:DD:EE:FF htf.Measurement("mac_address") .matches_regex(r"^([0-9A-F]{2}:){5}[0-9A-F]{2}$") .doc("Board MAC address format"), # Semantic version: X.Y.Z htf.Measurement("fw_version") .matches_regex(r"^\d+\.\d+\.\d+$") .doc("Firmware version format"), ) def test_strings(test): test.measurements.mac_address = "AA:BB:CC:DD:EE:FF" test.measurements.fw_version = "2.1.0" ``` ## Marginal Limits OpenHTF supports marginal limits (inner limits within the pass range). A measurement between the marginal limit and the spec limit passes but is flagged as marginal. This catches measurements that are drifting toward failure. ```python filename="measurements/marginal.py" import openhtf as htf from openhtf.util import units @htf.measures( htf.Measurement("rail_3v3") .in_range( minimum=3.2, maximum=3.4, marginal_minimum=3.22, marginal_maximum=3.38, ) .with_units(units.VOLT) .doc("3.3V rail with marginal band"), ) def test_marginal_voltage(test): # 3.21V passes but is flagged as marginal (between 3.20 and 3.22) # 3.25V passes normally (within marginal limits) # 3.19V fails (below 3.20) test.measurements.rail_3v3 = 3.21 ``` | Value | Status | Explanation | |-------|--------|-------------| | 3.25 | Pass | Within marginal limits | | 3.21 | Marginal | Between spec limit (3.20) and marginal limit (3.22) | | 3.39 | Marginal | Between marginal limit (3.38) and spec limit (3.40) | | 3.19 | Fail | Below spec limit (3.20) | | 3.41 | Fail | Above spec limit (3.40) | ## Units Reference OpenHTF provides standard SI units via `from openhtf.util import units`. Always use `.with_units()` for numeric measurements. It feeds into TofuPilot analytics and makes reports readable. | Unit | OpenHTF Constant | Use Case | |------|-----------------|----------| | Volt | `units.VOLT` | Voltage rails, analog signals | | Ampere | `units.AMPERE` | Current draw | | Ohm | `units.OHM` | Resistance, impedance | | Hertz | `units.HERTZ` | Frequency, clock signals | | Celsius | `units.DEGREE_CELSIUS` | Temperature | | Second | `units.SECOND` | Time, latency | | Watt | `units.WATT` | Power consumption | ## Multiple Measurements Per Phase Group related measurements in one phase. This keeps the test report organized and the test time efficient. ```python filename="measurements/multi.py" import openhtf as htf from openhtf.util import units @htf.measures( htf.Measurement("rail_3v3").in_range(3.2, 3.4).with_units(units.VOLT), htf.Measurement("rail_5v0").in_range(4.8, 5.2).with_units(units.VOLT), htf.Measurement("idle_current").in_range(0.05, 0.25).with_units(units.AMPERE), htf.Measurement("led_on").equals(True), htf.Measurement("firmware").equals("2.1.0"), ) def test_board_basics(test): test.measurements.rail_3v3 = 3.31 test.measurements.rail_5v0 = 5.01 test.measurements.idle_current = 0.12 test.measurements.led_on = True test.measurements.firmware = "2.1.0" ``` **Guideline:** Group measurements that are logically related (all power rails in one phase, all communication checks in another). Split into separate phases if they require different instruments or have different timeouts. ## Setting Limits from Datasheets Start with the datasheet absolute maximum and minimum ratings. Then tighten based on production data. | Source | Use For | Example | |--------|---------|---------| | Datasheet | Initial limits | 3.3V regulator: 3.135V to 3.465V (5% tolerance) | | Production data (100+ units) | Refined limits | 3-sigma from measured distribution | | Customer requirements | Mandatory limits | Specified in test specification | ```python filename="measurements/limits_from_datasheet.py" import openhtf as htf from openhtf.util import units # Datasheet says: 3.3V +/- 5% = 3.135V to 3.465V # Production data shows: mean=3.30, stdev=0.015 # 3-sigma: 3.255 to 3.345 (tighter, use these) @htf.measures( htf.Measurement("rail_3v3") .in_range(3.255, 3.345) .with_units(units.VOLT) .doc("3.3V rail (3-sigma from production data)"), ) def test_refined_limits(test): test.measurements.rail_3v3 = 3.31 ``` ### Automate a Rigol Power Supply URL: https://www.tofupilot.com/guides/how-to-automate-a-rigol-power-supply-with-python-and-tofupilot Control a Rigol DP800 series power supply from Python using PyVISA, with multi-channel control, OVP/OCP protection, and TofuPilot integration via OpenHTF. Control a Rigol DP800 series power supply over USB or Ethernet using Python, wrap it in an OpenHTF plug, and log results to TofuPilot. This guide covers connection, multi-channel control, protection settings, and a full production test example. ## Prerequisites - Rigol DP800 series PSU (DP821, DP831, DP832, or DP832A) - Python 3.8+ - USB-B cable or network connection to the PSU ```bash filename="terminal" pip install pyvisa pyvisa-py tofupilot openhtf ``` ## Step 1: Connect to the Instrument Rigol DP800 instruments expose a VISA interface over USB-TMC and TCP/IP. | Connection | VISA address example | |---|---| | USB-TMC | `USB0::0x1AB1::0x0E11::DP8C224200001::INSTR` | | Ethernet (VXI-11) | `TCPIP0::192.168.1.50::inst0::INSTR` | | Ethernet (raw socket) | `TCPIP0::192.168.1.50::5555::SOCKET` | ```python filename="connect_rigol.py" import pyvisa rm = pyvisa.ResourceManager() psu = rm.open_resource("USB0::0x1AB1::0x0E11::DP8C224200001::INSTR") psu.timeout = 5000 psu.write_termination = "\n" psu.read_termination = "\n" print(psu.query("*IDN?")) # Rigol Technologies,DP832,DP8C224200001,00.01.14 ``` ## Step 2: Understand Rigol SCPI Syntax Rigol DP800 SCPI differs from Keysight in a few important ways. | Operation | Rigol DP800 | Keysight E36xx | |---|---|---| | Select channel | `:INST CH1` | `:INST:SEL OUT1` | | Set voltage | `:VOLT 5.0` | `:VOLT 5.0` | | Set current limit | `:CURR 1.0` | `:CURR 1.0` | | Enable output | `:OUTP CH1,ON` | `:OUTP ON` | | Read voltage | `:MEAS:VOLT? CH1` | `:MEAS:VOLT?` | | Read current | `:MEAS:CURR? CH1` | `:MEAS:CURR?` | | OVP enable | `:OUTP:OVP CH1,ON` | `:VOLT:PROT:STAT ON` | | OVP level | `:OUTP:OVP:VAL CH1,5.5` | `:VOLT:PROT:LEV 5.5` | ## Step 3: Control Voltage, Current, and Output ```python filename="rigol_basic_control.py" import pyvisa rm = pyvisa.ResourceManager() psu = rm.open_resource("USB0::0x1AB1::0x0E11::DP8C224200001::INSTR") psu.timeout = 5000 def set_channel(psu, channel: int, voltage: float, current: float) -> None: psu.write(f":INST CH{channel}") psu.write(f":VOLT {voltage:.3f}") psu.write(f":CURR {current:.3f}") def enable_output(psu, channel: int) -> None: psu.write(f":OUTP CH{channel},ON") def disable_output(psu, channel: int) -> None: psu.write(f":OUTP CH{channel},OFF") def measure_voltage(psu, channel: int) -> float: return float(psu.query(f":MEAS:VOLT? CH{channel}")) def measure_current(psu, channel: int) -> float: return float(psu.query(f":MEAS:CURR? CH{channel}")) # Example: power a 3.3 V rail at up to 500 mA set_channel(psu, 1, voltage=3.3, current=0.5) enable_output(psu, 1) vout = measure_voltage(psu, 1) iout = measure_current(psu, 1) print(f"CH1: {vout:.3f} V, {iout:.4f} A") ``` ## Step 4: Multi-Channel Control (DP832) The DP832 has three independent channels. Configure all channels before enabling any output. ```python filename="rigol_multichannel.py" import pyvisa import time rm = pyvisa.ResourceManager() psu = rm.open_resource("USB0::0x1AB1::0x0E11::DP8C224200001::INSTR") psu.timeout = 5000 CHANNELS = { 1: {"voltage": 5.0, "current": 2.0}, # Main 5 V rail 2: {"voltage": 3.3, "current": 1.0}, # Logic rail 3: {"voltage": 12.0, "current": 0.5}, # Analog rail } for ch, cfg in CHANNELS.items(): psu.write(f":INST CH{ch}") psu.write(f":VOLT {cfg['voltage']:.3f}") psu.write(f":CURR {cfg['current']:.3f}") for ch in CHANNELS: psu.write(f":OUTP CH{ch},ON") time.sleep(0.1) for ch in CHANNELS: v = float(psu.query(f":MEAS:VOLT? CH{ch}")) i = float(psu.query(f":MEAS:CURR? CH{ch}")) print(f"CH{ch}: {v:.3f} V {i:.4f} A") ``` ## Step 5: Configure OVP and OCP Protection Always set protection before enabling output in a production environment. ```python filename="rigol_protection.py" def configure_protection(psu, channel: int, ovp_volts: float, ocp_amps: float) -> None: psu.write(f":OUTP:OVP:VAL CH{channel},{ovp_volts:.3f}") psu.write(f":OUTP:OVP CH{channel},ON") psu.write(f":OUTP:OCP:VAL CH{channel},{ocp_amps:.3f}") psu.write(f":OUTP:OCP CH{channel},ON") # 5 V rail: trip at 5.5 V or 2.5 A configure_protection(psu, 1, ovp_volts=5.5, ocp_amps=2.5) ``` OVP and OCP trip levels should be set 10-15% above nominal to avoid nuisance trips while still catching faults quickly. ## Step 6: Build an OpenHTF Plug ```python filename="plugs/rigol_dp800.py" import time import pyvisa import openhtf as htf class RigolDP800(htf.plugs.BasePlug): """OpenHTF plug for Rigol DP800 series power supplies.""" VISA_ADDRESS = "USB0::0x1AB1::0x0E11::DP8C224200001::INSTR" def setUp(self): self._rm = pyvisa.ResourceManager() self._psu = self._rm.open_resource(self.VISA_ADDRESS) self._psu.timeout = 5000 self._psu.write_termination = "\n" self._psu.read_termination = "\n" def tearDown(self): for ch in (1, 2, 3): try: self._psu.write(f":OUTP CH{ch},OFF") except Exception: pass if self._psu: self._psu.close() self._rm.close() def configure(self, channel: int, voltage: float, current: float): self._psu.write(f":INST CH{channel}") self._psu.write(f":VOLT {voltage:.3f}") self._psu.write(f":CURR {current:.3f}") def set_protection(self, channel: int, ovp: float, ocp: float): self._psu.write(f":OUTP:OVP:VAL CH{channel},{ovp:.3f}") self._psu.write(f":OUTP:OVP CH{channel},ON") self._psu.write(f":OUTP:OCP:VAL CH{channel},{ocp:.3f}") self._psu.write(f":OUTP:OCP CH{channel},ON") def enable(self, channel: int): self._psu.write(f":OUTP CH{channel},ON") def disable(self, channel: int): self._psu.write(f":OUTP CH{channel},OFF") def measure_voltage(self, channel: int) -> float: return float(self._psu.query(f":MEAS:VOLT? CH{channel}")) def measure_current(self, channel: int) -> float: return float(self._psu.query(f":MEAS:CURR? CH{channel}")) ``` ## Step 7: Write the Production Test with TofuPilot ```python filename="test_power_supply.py" import time import openhtf as htf from openhtf.util import units from tofupilot.openhtf import TofuPilot from plugs.rigol_dp800 import RigolDP800 @htf.plug(psu=RigolDP800) def power_on_dut(test, psu): """Configure rails, enable protection, and power the DUT.""" psu.configure(channel=1, voltage=5.0, current=2.0) psu.set_protection(channel=1, ovp=5.5, ocp=2.5) psu.configure(channel=2, voltage=3.3, current=1.0) psu.set_protection(channel=2, ovp=3.7, ocp=1.2) psu.enable(channel=1) psu.enable(channel=2) time.sleep(0.5) @htf.plug(psu=RigolDP800) @htf.measures( htf.Measurement("ch1_voltage") .in_range(minimum=4.85, maximum=5.15) .with_units(units.VOLT), htf.Measurement("ch1_current") .in_range(maximum=2.0) .with_units(units.AMPERE), htf.Measurement("ch2_voltage") .in_range(minimum=3.20, maximum=3.40) .with_units(units.VOLT), htf.Measurement("ch2_current") .in_range(maximum=1.0) .with_units(units.AMPERE), ) def measure_power_rails(test, psu): """Read and validate all supply rails under load.""" test.measurements.ch1_voltage = psu.measure_voltage(channel=1) test.measurements.ch1_current = psu.measure_current(channel=1) test.measurements.ch2_voltage = psu.measure_voltage(channel=2) test.measurements.ch2_current = psu.measure_current(channel=2) @htf.plug(psu=RigolDP800) def power_off_dut(test, psu): """Disable all outputs after test.""" psu.disable(channel=1) psu.disable(channel=2) def main(): test = htf.Test( power_on_dut, measure_power_rails, power_off_dut, test_name="Power Supply Validation", ) with TofuPilot(test): test.execute(test_start=lambda: input("Enter serial number: ").strip()) if __name__ == "__main__": main() ``` ## Troubleshooting | Symptom | Likely Cause | Fix | |---------|-------------|-----| | Instrument not found on USB | Missing udev rules (Linux) | Add `SUBSYSTEM=="usb", ATTR{idVendor}=="1ab1", MODE="0666"` to udev | | `VI_ERROR_TMO` on query | `read_termination` not set | Set `psu.read_termination = "\n"` | | Output trips OCP immediately | Capacitive inrush exceeds limit | Raise OCP limit 20% above steady-state or ramp voltage | | Measurements read 0 V | Channel in CC mode, load too heavy | Increase current limit or reduce load | | Ethernet connection drops | Idle VXI-11 timeout (~60s) | Re-open resource or send periodic `*IDN?` keep-alive | | Command returns no data | Used `query()` for set command | Use `write()` for commands that don't return data | ### What Is an Operator Interface URL: https://www.tofupilot.com/guides/what-is-an-operator-interface-for-manufacturing-test An operator interface lets production floor workers run tests without writing code. Learn what it includes, how it compares across frameworks, and how to. # What Is an Operator Interface for Manufacturing Test An operator interface is the screen a production floor worker sees when running a test. It handles serial number entry, displays pass/fail results, shows prompts for manual steps, and logs everything without the operator touching code. This guide covers what an operator interface should include, how the major test frameworks handle it, and how to set one up with OpenHTF and TofuPilot. ## Why Operators Need a Dedicated Interface Test engineers write Python scripts. Operators run them. These are different jobs with different needs. | Test Engineer | Operator | |--------------|----------| | Writes and debugs test code | Runs tests hundreds of times per shift | | Reads terminal output | Needs large pass/fail indicators | | Knows what each phase does | Needs step-by-step instructions | | Has Python installed | May not know what Python is | | Works on one station | May rotate between stations | Running a test from a terminal works during development. In production, the operator needs a purpose-built screen that removes complexity and prevents mistakes. ## What an Operator Interface Should Include | Feature | Why | |---------|-----| | Serial number input | Barcode scanner or manual entry to identify each DUT | | Start/stop controls | One button to begin, one to abort | | Pass/fail display | Large, color-coded result visible from a distance | | Phase progress | Shows which step is running and how many remain | | Operator prompts | Asks for manual actions (flip board, connect cable, visual check) | | Input fields | Captures operator observations (dropdown, checkbox, text, numeric) | | Measurement display | Shows live readings during the test | | Error messages | Clear instructions when something goes wrong | | Test history | Last few results for the current station | ## How Test Frameworks Handle Operator Interfaces ### NI TestStand TestStand ships with pre-built operator interfaces in LabVIEW, C#, and VB.NET. The source code is provided so you can customize it. It supports user roles (operator, developer, admin), concurrent test executions, and report generation. **Strengths:** Mature, full-featured, customizable. **Weaknesses:** Requires a TestStand license ($3-5K per seat), Windows only, tied to the NI ecosystem. ### OpenTAP OpenTAP offers an operator panel plugin that provides a simplified view for running test plans. It includes a "focus mode" that minimizes interactions to start/stop and pass/fail. **Strengths:** Open source core, cross-platform. **Weaknesses:** .NET ecosystem, sparse documentation, basic operator UI. ### OpenHTF OpenHTF includes a built-in Station Server that serves a web interface at `localhost:12000`. It shows test status, phase progress, and handles operator prompts through the browser. **Strengths:** Web-based (works on any device with a browser), no installation needed on the operator's machine. **Weaknesses:** The built-in UI is minimal. For production use, most teams pair OpenHTF with a dedicated frontend. ### pytest with HardPy HardPy adds a browser-based operator panel to pytest. It shows test hierarchy, dialog boxes for operator input, and real-time charts. **Strengths:** pytest-native, modern web UI. **Weaknesses:** New project, limited adoption, basic feature set. ## Comparison Table | Feature | TestStand | OpenTAP | OpenHTF + TofuPilot | HardPy | |---------|-----------|---------|---------------------|--------| | Serial number input | Yes | Yes | Yes | Yes | | Pass/fail display | Yes | Yes | Yes (streaming) | Yes | | Operator prompts | Yes | Limited | Yes (15+ input types) | Yes (dialogs) | | Image-based inputs | No | No | Yes | No | | Real-time streaming | Yes | Limited | Yes (MQTT) | Yes | | Role management | Yes | No | Yes | No | | Web-based | No (desktop) | No (desktop) | Yes | Yes | | Kiosk mode | Manual setup | No | Yes | No | | Cost | $3-5K/seat | Free (core) | Free (Lab tier) | Free | | Platform | Windows | Windows/Linux | Any browser | Any browser | ## Prerequisites - Python 3.10+ - OpenHTF installed (`pip install openhtf`) - TofuPilot Python SDK installed (`pip install tofupilot`) ## Step 1: Add Operator Prompts to Your Test OpenHTF handles operator interaction through prompts. The operator sees a message and optionally provides input before the test continues. ```python filename="operator_test.py" import openhtf as htf from openhtf.plugs import user_input @htf.plug(prompts=user_input.UserInput) def phase_visual_inspection(test, prompts): """Ask the operator to perform a visual check.""" prompts.prompt( "Inspect the board for solder bridges or missing components. " "Press Enter when done." ) @htf.plug(prompts=user_input.UserInput) @htf.measures( htf.Measurement("operator_serial").with_args(docstring="Scanned serial number"), ) def phase_scan_serial(test, prompts): """Prompt the operator to scan the DUT serial number.""" serial = prompts.prompt( "Scan the barcode on the DUT label.", text_input=True, ) test.measurements.operator_serial = serial ``` ## Step 2: Stream to TofuPilot Operator UI TofuPilot provides a web-based operator interface that streams test progress in real time. The operator sees prompts, measurements, and pass/fail results in a browser. No terminal, no code. ```python filename="operator_test.py" from tofupilot.openhtf import TofuPilot test = htf.Test( phase_scan_serial, phase_visual_inspection, ) with TofuPilot(test): test.execute(test_start=lambda: input("Scan serial: ")) ``` When the test runs, TofuPilot streams the interface to a URL the operator can open on any device. The console prints the URL automatically. ## Step 3: Deploy for Production For production use, set up the operator station so the browser opens automatically in full-screen mode: | Setting | Recommendation | |---------|---------------| | Browser | Chrome or Edge in kiosk mode (full screen, no address bar) | | Display | Large monitor or touchscreen at the test station | | Input | Barcode scanner configured as keyboard input | | Font size | Browser zoom to 125-150% for readability at arm's length | | Auto-start | Launch browser on boot, navigate to the TofuPilot streaming URL | The operator's workflow becomes: scan barcode, follow prompts, see pass/fail. No terminal, no file system, no Python knowledge required. ## Choosing the Right Approach | Situation | Recommendation | |-----------|---------------| | Already using TestStand | Use the built-in OI unless you're migrating away | | Python-based tests, need a quick UI | OpenHTF + TofuPilot streaming (no frontend code to write) | | pytest-based tests | HardPy for basic prompts, or TofuPilot for full operator UI | | Custom requirements (branded, multi-station dashboard) | Build a custom frontend against the TofuPilot API | | Budget is zero, needs to work today | OpenHTF Station Server at localhost:12000 | ### Track Equipment Calibration URL: https://www.tofupilot.com/guides/how-to-track-equipment-calibration-with-tofupilot Validate instrument calibration status before every test run. Track calibration dates as metadata and catch expired calibrations before they corrupt your data. Test instruments drift over time. A multimeter that was accurate six months ago might now read 50mV high, pushing borderline units past their limits. Tracking calibration status in your test workflow prevents expired instruments from corrupting your production data. ## Why Calibration Tracking Matters Regulatory frameworks like ISO 17025 and ISO 9001 require traceable calibration for test equipment. But compliance aside, the practical reason is simple: if your instruments aren't accurate, your pass/fail decisions aren't reliable. An uncalibrated power supply might output 5.08V when it claims 5.00V. A DMM with drift might read 3.28V when the true value is 3.31V. These errors shift your measurement distributions and can cause both false passes (shipping bad units) and false failures (scrapping good ones). ## Validate Calibration Before Testing Build a calibration check into your test sequence. Before running any DUT measurements, query the instrument's calibration date and compare it against the expiration window. ```python filename="test_with_cal_check.py" import openhtf as htf from openhtf.util import units from tofupilot.openhtf import TofuPilot from datetime import datetime, timedelta CALIBRATION_INTERVAL_DAYS = 180 @htf.measures( htf.Measurement("dmm_cal_days_remaining") .in_range(minimum=0) .doc("Days until DMM calibration expires. Fails if expired.") ) def check_dmm_calibration(test): # Read last calibration date from instrument or local config last_cal_date = datetime(2026, 1, 15) expiry_date = last_cal_date + timedelta(days=CALIBRATION_INTERVAL_DAYS) days_remaining = (expiry_date - datetime.now()).days test.measurements.dmm_cal_days_remaining = days_remaining @htf.measures( htf.Measurement("output_voltage") .in_range(minimum=3.25, maximum=3.35) .with_units(units.VOLT), htf.Measurement("ripple") .in_range(maximum=50.0), ) def test_voltage_regulator(test): test.measurements.output_voltage = 3.30 test.measurements.ripple = 12.4 def main(): test = htf.Test( check_dmm_calibration, test_voltage_regulator, station_id="SZ-L1-FCT-03", ) with TofuPilot(test): test.execute(test_start=lambda: "REG-2026-08841") if __name__ == "__main__": main() ``` The `check_dmm_calibration` phase runs first. If the calibration has expired (days remaining is negative), the measurement fails its `minimum=0` limit and the entire test run fails. No DUT measurements are taken with an out-of-cal instrument. ## Track Multiple Instruments Per Station Most test stations use several instruments. Check each one. ```python filename="test_with_multi_cal.py" import openhtf as htf from tofupilot.openhtf import TofuPilot from datetime import datetime, timedelta CAL_INTERVAL = timedelta(days=180) def days_until_expiry(last_cal_date): return (last_cal_date + CAL_INTERVAL - datetime.now()).days @htf.measures( htf.Measurement("cal_dmm_days_remaining").in_range(minimum=0), htf.Measurement("cal_scope_days_remaining").in_range(minimum=0), htf.Measurement("cal_psu_days_remaining").in_range(minimum=0), ) def check_all_calibrations(test): # Read from instrument memory, config file, or calibration database test.measurements.cal_dmm_days_remaining = days_until_expiry(datetime(2026, 1, 15)) test.measurements.cal_scope_days_remaining = days_until_expiry(datetime(2025, 11, 20)) test.measurements.cal_psu_days_remaining = days_until_expiry(datetime(2026, 2, 1)) def main(): test = htf.Test( check_all_calibrations, station_id="GDL-L2-FCT-01", ) with TofuPilot(test): test.execute(test_start=lambda: "UNIT-2026-12003") if __name__ == "__main__": main() ``` Each instrument gets its own measurement. If the oscilloscope's calibration expires but the DMM and PSU are fine, TofuPilot shows exactly which instrument needs attention. ## Store Calibration Metadata Beyond the pass/fail check, record calibration details so you have a full audit trail. ```python filename="test_with_cal_metadata.py" import openhtf as htf from openhtf.util import units from tofupilot.openhtf import TofuPilot @htf.measures( htf.Measurement("cal_dmm_days_remaining").in_range(minimum=0), ) def check_calibration(test): test.measurements.cal_dmm_days_remaining = 47 @htf.measures( htf.Measurement("supply_voltage") .in_range(minimum=11.8, maximum=12.2) .with_units(units.VOLT), ) def test_supply(test): test.measurements.supply_voltage = 12.03 def main(): test = htf.Test( check_calibration, test_supply, station_id="AUS-L1-EOL-02", ) with TofuPilot(test): test.execute(test_start=lambda: "PSU-2026-00219") if __name__ == "__main__": main() ``` TofuPilot stores every run with its full measurement data. If a quality audit asks "was the DMM calibrated when unit PSU-2026-00219 was tested?", you can answer definitively by looking up that run. ## Use Fixture Validation Phases Beyond instrument calibration, test fixtures themselves need validation. Add a fixture check phase that measures a known reference standard before testing DUTs. ```python filename="test_with_fixture_check.py" import openhtf as htf from openhtf.util import units from tofupilot.openhtf import TofuPilot @htf.measures( htf.Measurement("fixture_ref_resistance") .in_range(minimum=99.5, maximum=100.5) .with_units(units.OHM) .doc("Golden reference resistor. Validates fixture contacts and DMM.") ) def validate_fixture(test): # Measure a known 100 ohm reference on the fixture test.measurements.fixture_ref_resistance = 100.1 @htf.measures( htf.Measurement("dut_resistance") .in_range(minimum=45.0, maximum=55.0) .with_units(units.OHM), ) def test_dut_resistance(test): test.measurements.dut_resistance = 49.7 def main(): test = htf.Test( validate_fixture, test_dut_resistance, station_id="SZ-L3-ICT-01", ) with TofuPilot(test): test.execute(test_start=lambda: "HTR-2026-06650") if __name__ == "__main__": main() ``` The fixture validation phase measures a golden reference. If the reading is outside tolerance, the fixture contacts are worn, the DMM has drifted, or both. Either way, the test stops before producing unreliable DUT data. ## Monitor Calibration Health in TofuPilot TofuPilot's measurement trends show your calibration health over time: - **Track `cal_*_days_remaining` trends** to see when instruments are approaching expiration across all stations - **Watch fixture validation measurements** for slow drift that indicates contact wear - **Filter by station** to see which specific machines need calibration soon - **Set up alerts** for when calibration days remaining drops below a threshold (e.g., 14 days) so you can schedule recalibration proactively ### What Is IQC, IPQC, FQC, and OQC URL: https://www.tofupilot.com/guides/what-is-iqc-ipqc-fqc-and-oqc-with-tofupilot IQC, IPQC, FQC, and OQC are quality control stages in manufacturing. Learn what each covers and how to track quality data with TofuPilot. # What Is IQC, IPQC, FQC, and OQC with TofuPilot Manufacturing quality control happens at four stages: incoming (IQC), in-process (IPQC), final (FQC), and outgoing (OQC). Each stage catches different types of defects at different costs. This guide covers what each stage involves, what it catches, and how to track quality data across all stages with TofuPilot. ## The Four Quality Gates | Stage | Full Name | When | What It Catches | |-------|-----------|------|----------------| | IQC | Incoming Quality Control | Materials arrive | Bad components, wrong parts, supplier issues | | IPQC | In-Process Quality Control | During manufacturing | Process drift, assembly errors, contamination | | FQC | Final Quality Control | After assembly complete | Functional failures, cosmetic defects | | OQC | Outgoing Quality Control | Before shipment | Packaging damage, labeling errors, sampling verification | Each stage is a gate. Material that fails a gate gets quarantined, reworked, or rejected. Material that passes moves to the next stage. ## IQC: Incoming Quality Control IQC inspects raw materials and components when they arrive from suppliers. It prevents bad inputs from entering the production line. | Check | Method | |-------|--------| | Visual inspection | Packaging damage, correct labels, moisture indicators | | Dimensional check | Caliper or CMM measurement against drawing | | Electrical sample test | Measure key parameters on a sample (AQL-based) | | Certificate review | Verify CoC, RoHS compliance, lot codes | IQC sampling follows AQL (Acceptable Quality Limit) tables. You don't test every component. You test a statistically valid sample and accept or reject the lot. ## IPQC: In-Process Quality Control IPQC monitors the manufacturing process while it's running. It catches drift before it produces a batch of defective units. | Check | Method | |-------|--------| | SPI after paste printing | Automated solder paste volume measurement | | AOI after reflow | Automated optical inspection of solder joints | | First piece inspection | Full check of the first unit after setup | | Hourly spot checks | Operator verifies key dimensions or parameters | | Process parameter monitoring | Temperature, pressure, speed within control limits | IPQC is where SPC (statistical process control) lives. Control charts track process parameters in real time and flag out-of-control conditions before they produce defects. ## FQC: Final Quality Control FQC tests the finished product against its full specification. This is where functional testing, safety testing, and cosmetic inspection happen. | Check | Method | |-------|--------| | Functional test | Automated test (ATE) against design spec | | Safety test | Hipot, ground continuity, leakage current | | Cosmetic inspection | Visual check for scratches, dents, label alignment | | Calibration verification | Confirm calibrated parameters are within spec | FQC is typically 100% inspection for critical tests and sample-based for cosmetic checks. ## OQC: Outgoing Quality Control OQC is the final check before the product ships. It verifies that packing, labeling, and quantity are correct, and runs a final sample test. | Check | Method | |-------|--------| | Packaging integrity | Box condition, cushioning, moisture barrier | | Label verification | Part number, serial, lot, regulatory marks | | Quantity count | Matches packing list and order | | Sample retest | Pull samples and rerun functional test | OQC is the last chance to catch problems. A failure here is expensive because the product is fully packaged and ready to ship. ## Prerequisites - Python 3.10+ - OpenHTF installed (`pip install openhtf`) - TofuPilot Python SDK installed (`pip install tofupilot`) ## Step 1: Log Quality Checks at Each Stage Create separate test procedures for each quality stage. This keeps the data organized and lets you track yield per stage independently. ```python filename="iqc_check.py" import openhtf as htf from openhtf.util import units @htf.measures( htf.Measurement("component_resistance_ohm") .in_range(minimum=4700, maximum=5100) .with_units(units.OHM), htf.Measurement("visual_inspection").equals("PASS"), ) def phase_iqc_sample(test): """IQC sample test for incoming resistor lot.""" test.measurements.component_resistance_ohm = 4850 test.measurements.visual_inspection = "PASS" ``` ```python filename="fqc_test.py" import openhtf as htf from openhtf.util import units @htf.measures( htf.Measurement("output_voltage_V") .in_range(minimum=4.9, maximum=5.1) .with_units(units.VOLT), htf.Measurement("safety_hipot").equals("PASS"), ) def phase_fqc(test): """FQC: functional and safety test on finished unit.""" test.measurements.output_voltage_V = 5.01 test.measurements.safety_hipot = "PASS" ``` ## Step 2: Connect Each Stage to TofuPilot Each quality stage uploads results independently. TofuPilot links them by serial number, giving you full traceability from incoming materials to outgoing product. ```python filename="fqc_test.py" from tofupilot.openhtf import TofuPilot test = htf.Test(phase_fqc) with TofuPilot(test): test.execute(test_start=lambda: input("Scan unit serial: ")) ``` ## Step 3: Track Quality Across All Stages TofuPilot tracks results from all quality stages. Open the Analytics tab to see: - **Yield per stage** (IQC, IPQC, FQC, OQC separately) - **Failure Pareto** per stage showing top defect types - **Supplier quality** by tracking IQC rejection rates per vendor - **Full traceability** from incoming lot to shipped unit ### What Is Design Validation Testing URL: https://www.tofupilot.com/guides/what-is-design-validation-testing-with-tofupilot Design validation testing (DVT) confirms a product meets its design requirements. Learn how to structure DVT in Python and track results with TofuPilot. # What Is Design Validation Testing with TofuPilot Design validation testing (DVT) confirms that a product meets its design requirements under real-world conditions. It happens after prototyping and before production tooling. This guide explains where DVT fits in the product development cycle, how to build DVT scripts in Python, and how to track validation results with TofuPilot. ## Where DVT Fits Hardware products typically go through three build stages: | Stage | Units | Purpose | |-------|-------|---------| | EVT (Engineering Validation) | 5-20 | Does the concept work? | | DVT (Design Validation) | 20-100 | Does the design meet requirements? | | PVT (Production Validation) | 100-500 | Can manufacturing build it reliably? | DVT answers a specific question: does this design meet the requirements in the spec? You test against the product requirements document (PRD), not against manufacturing tolerances. The units are built on near-final tooling, and the test conditions include environmental extremes, mechanical stress, and electrical corner cases. ## What DVT Covers | Category | Example Tests | |----------|--------------| | Electrical | Power consumption, signal integrity, EMI/EMC | | Mechanical | Drop, vibration, torque, insertion force | | Environmental | Temperature cycling, humidity, altitude | | Reliability | HALT, accelerated life testing, endurance | | Safety | Leakage current, dielectric strength, flammability | | Functional | All user-facing features under nominal and edge conditions | DVT is not pass/fail in the EOL sense. It produces data that feeds design decisions. A measurement that lands close to a limit might pass DVT but prompt a design change to add margin before PVT. ## Prerequisites - Python 3.10+ - OpenHTF installed (`pip install openhtf`) - TofuPilot Python SDK installed (`pip install tofupilot`) ## Step 1: Define Validation Measurements DVT phases often measure the same parameter under different conditions. Use descriptive measurement names that include the test condition. ```python filename="dvt_thermal.py" import openhtf as htf from openhtf.util import units @htf.measures( htf.Measurement("current_draw_25C_mA") .in_range(minimum=40, maximum=60) .with_units(units.MILLIAMPERE), htf.Measurement("current_draw_85C_mA") .in_range(minimum=40, maximum=70) .with_units(units.MILLIAMPERE), ) def phase_thermal_current(test): """Measure current draw at room temp and high temp.""" test.measurements.current_draw_25C_mA = 48.3 test.measurements.current_draw_85C_mA = 55.1 ``` ## Step 2: Add Marginal Limits DVT benefits from marginal limits. A measurement inside the marginal band passes but flags a potential risk. TofuPilot tracks marginal results separately so you can spot trends before they become failures. ```python filename="dvt_thermal.py" @htf.measures( htf.Measurement("output_voltage_V") .in_range( minimum=3.0, maximum=3.6, marginal_minimum=3.1, marginal_maximum=3.5, ) .with_units(units.VOLT), ) def phase_output_voltage(test): """Validate output voltage is within spec with margin.""" test.measurements.output_voltage_V = 3.28 ``` ## Step 3: Run and Track Results Connect the DVT script to TofuPilot. Each unit in the DVT build gets its own test record, making it easy to compare performance across the build lot. ```python filename="dvt_thermal.py" from tofupilot.openhtf import TofuPilot test = htf.Test( phase_thermal_current, phase_output_voltage, ) with TofuPilot(test): test.execute(test_start=lambda: input("Scan DVT unit serial: ")) ``` ## Step 4: Analyze in TofuPilot TofuPilot tracks DVT results automatically. Open the Analytics tab to review: - **Measurement distributions** across the DVT build lot - **Marginal results** flagged separately from hard failures - **Unit-level traceability** linking each prototype to its full test history - **Trends over time** to catch drift across sequential builds This data feeds directly into the DVT report. Instead of manually compiling spreadsheets, export the data from TofuPilot or share the dashboard link with your team. ## DVT vs Production Testing | Aspect | DVT | Production Test | |--------|-----|-----------------| | Goal | Verify the design | Verify each unit | | Sample size | 20-100 units | Every unit | | Test depth | Deep, multi-condition | Fast, single-condition | | Limits | Spec-derived, with margin bands | Tightened from production data | | Output | Design report | Ship/no-ship decision | DVT limits often start wider and tighten as you move into PVT and production. TofuPilot lets you track this evolution by comparing measurement distributions across test procedures for the same product. ### Test Station Monitoring and Performance URL: https://www.tofupilot.com/guides/test-station-monitoring-track-uptime-and-performance-with-tofupilot Learn how to monitor test station health, track uptime metrics, and detect performance degradation using TofuPilot analytics. A test station that's silently degrading is worse than one that's down. Slow tests eat throughput. Dropping yields hide root causes. You need visibility into station health before problems hit your production line. This guide covers what to monitor, how to capture station health metrics inside your OpenHTF tests, and how to detect drift using TofuPilot's analytics. ## What to Monitor Four metrics tell you most of what you need to know about a test station. **Test throughput.** Units per hour, per station. A drop means something changed: slower tests, more retests, or operator delays. **Pass rate trends.** FPY over time, not just today's number. A slow decline from 97% to 93% over two weeks is easy to miss in daily reports. **Average test duration.** Track per-phase and total. If your calibration phase went from 4s to 12s, the instrument connection is probably degrading. **Station errors.** Uncaught exceptions, instrument timeouts, fixture faults. These don't always fail the DUT, but they signal trouble. ## Capture Station Health Metrics You can log station health data (CPU, memory, disk) as OpenHTF measurements alongside your DUT tests. This gives you a per-run snapshot of station condition. ```python filename="station_health.py" import openhtf as htf import psutil @htf.measures( htf.Measurement("cpu_percent").in_range(maximum=90), htf.Measurement("memory_percent").in_range(maximum=85), htf.Measurement("disk_percent").in_range(maximum=90), htf.Measurement("disk_read_mb"), htf.Measurement("cpu_temp"), ) def station_health_check(test): """Capture station health metrics before running DUT tests.""" test.measurements.cpu_percent = psutil.cpu_percent(interval=1) test.measurements.memory_percent = psutil.virtual_memory().percent test.measurements.disk_percent = psutil.disk_usage("/").percent test.measurements.disk_read_mb = psutil.disk_io_counters().read_bytes / (1024 * 1024) # CPU temperature (Linux only, returns empty list on other platforms) temps = psutil.sensors_temperatures() if temps and "coretemp" in temps: test.measurements.cpu_temp = temps["coretemp"][0].current else: test.measurements.cpu_temp = 0.0 ``` Add this phase at the start of your test sequence. If CPU or memory is pegged, you'll see it in TofuPilot before it causes flaky test results. ```python filename="main_test.py" import openhtf as htf from openhtf.util import units from tofupilot.openhtf import TofuPilot from station_health import station_health_check # Your DUT test phases @htf.measures(htf.Measurement("voltage_3v3").in_range(3.1, 3.5).with_units(units.VOLT)) def test_power_rail(test): test.measurements.voltage_3v3 = 3.28 def main(): test = htf.Test( station_health_check, test_power_rail, ) with TofuPilot(test): test.execute(test_start=lambda: "DUT-001") if __name__ == "__main__": main() ``` ## Detect Performance Drift in TofuPilot TofuPilot tracks test duration, pass rates, and measurement trends per station automatically. Use the Analytics tab to spot drift: - **Test duration trend.** Filter by station and check whether average test time is increasing. A 15%+ increase over baseline signals instrument connection degradation, fixture wear, or background process interference. - **FPY by station.** Compare yield across stations running the same procedure. A station with 3+ points lower FPY than its neighbors needs fixture inspection. - **Measurement histograms.** Check whether station health measurements (CPU, memory, disk) are creeping toward their limits over time. - **Failure Pareto.** If one station accounts for a disproportionate share of failures, investigate that station's fixture and connections. ## Monitoring Checklist | Metric | Frequency | Threshold | Action | |--------|-----------|-----------|--------| | Test throughput (units/hr) | Hourly | Below 80% of target | Check for operator delays, instrument timeouts | | First pass yield | Per shift | Below 95% (or your target) | Investigate top failing phases | | Average test duration | Daily | More than 15% above baseline | Check instrument connections, fixture wear | | CPU usage | Per run | Above 90% | Close background processes, check for memory leaks | | Memory usage | Per run | Above 85% | Restart station, check for leaking test processes | | Disk usage | Daily | Above 90% | Clean logs, archive old data | | Station errors | Per run | Any uncaught exception | Fix root cause, add error handling | | Instrument timeout rate | Daily | Above 1% | Check cables, GPIB/USB connections | ### Set Measurement Limits from Prod Data URL: https://www.tofupilot.com/guides/how-to-set-measurement-limits-using-production-data-with-tofupilot Learn how to derive statistically sound measurement limits from production data using the 3-sigma method, detect distribution anomalies, and configure. Datasheet limits are starting points, not production limits. Real boards have tolerances, assembly variation, and environmental drift that datasheets cannot capture. This guide shows you how to collect measurement data from your production line, apply the 3-sigma method to derive statistically sound limits, and configure those limits in OpenHTF with TofuPilot tracking. ## Prerequisites - TofuPilot account with production run data - Python 3.8+ - `numpy` and `scipy` installed - At least 30 test samples (100+ recommended) ## Why Datasheet Limits Are Not Enough | Source of Variation | Example | |---------------------|---------| | Component tolerance | Resistor +/-5% shifts output voltage | | PCB trace impedance | Layout differences across panel positions | | Assembly variation | Solder joint resistance changes | | Environmental drift | Temperature at test station vs. field | | Measurement noise | Probe contact, cable length | A 3.3V rail with a datasheet range of +/-5% (3.135V-3.465V) may show actual production distribution centered at 3.31V with a std of 0.012V. Setting limits from the datasheet alone misses that your process is drifting high. ## Step 1: Collect Baseline Samples You need a minimum of 30 samples from boards you have already validated as good. Use 100+ for stable sigma estimates. ```python filename="collect_samples.py" import numpy as np # Simulated collection from known-good boards # In practice, export from TofuPilot or accumulate from live runs vdd_samples = [ 3.312, 3.308, 3.315, 3.310, 3.307, 3.319, 3.311, 3.314, 3.309, 3.316, 3.313, 3.308, 3.312, 3.317, 3.310, 3.306, 3.315, 3.311, 3.313, 3.309, 3.318, 3.310, 3.312, 3.315, 3.308, 3.311, 3.314, 3.307, 3.316, 3.313, 3.310, 3.308, 3.315, 3.312, 3.311, 3.309, 3.317, 3.314, 3.310, 3.313, ] n = len(vdd_samples) mean = np.mean(vdd_samples) std = np.std(vdd_samples, ddof=1) # sample std, not population print(f"N={n} mean={mean:.4f}V std={std:.4f}V") # N=40 mean=3.3119V std=0.0031V ``` Use `ddof=1` for sample standard deviation when your sample size is less than the full population. ## Step 2: Check for Outliers and Bimodal Distributions Before computing limits, inspect the distribution. A bimodal distribution signals two distinct populations. ```python filename="check_distribution.py" import numpy as np from scipy import stats vdd_samples = [ 3.312, 3.308, 3.315, 3.310, 3.307, 3.319, 3.311, 3.314, 3.309, 3.316, 3.313, 3.308, 3.312, 3.317, 3.310, 3.306, 3.315, 3.311, 3.313, 3.309, 3.318, 3.310, 3.312, 3.315, 3.308, 3.311, 3.314, 3.307, 3.316, 3.313, 3.310, 3.308, 3.315, 3.312, 3.311, 3.309, 3.317, 3.314, 3.310, 3.313, ] # Z-score outlier detection z_scores = np.abs(stats.zscore(vdd_samples)) outliers = [v for v, z in zip(vdd_samples, z_scores) if z > 3] print(f"Outliers (|z| > 3): {outliers}") # Shapiro-Wilk normality test stat, p = stats.shapiro(vdd_samples) print(f"Shapiro-Wilk p={p:.4f} ({'normal' if p > 0.05 else 'NOT normal'})") ``` If Shapiro-Wilk returns `p < 0.05`, investigate whether your samples came from two assembly batches or two component reels. ## Step 3: Compute 3-Sigma Limits The 3-sigma rule covers 99.73% of a normal distribution (roughly 2700 DPMO at the limit edge). ```python filename="compute_limits.py" import numpy as np vdd_samples = [ 3.312, 3.308, 3.315, 3.310, 3.307, 3.319, 3.311, 3.314, 3.309, 3.316, 3.313, 3.308, 3.312, 3.317, 3.310, 3.306, 3.315, 3.311, 3.313, 3.309, 3.318, 3.310, 3.312, 3.315, 3.308, 3.311, 3.314, 3.307, 3.316, 3.313, 3.310, 3.308, 3.315, 3.312, 3.311, 3.309, 3.317, 3.314, 3.310, 3.313, ] mean = np.mean(vdd_samples) std = np.std(vdd_samples, ddof=1) sigma_levels = { "3s (99.73%)": 3, "4s (99.9937%)": 4, "6s (99.99966%)": 6, } print(f"Mean: {mean:.4f} V") print(f"Std: {std:.4f} V\n") for label, k in sigma_levels.items(): lo = mean - k * std hi = mean + k * std print(f"{label:25s} low={lo:.4f}V high={hi:.4f}V") ``` | Sigma Level | Coverage | Hard limits (VDD example) | |-------------|----------|---------------------------| | 3-sigma | 99.73% | 3.3026V to 3.3212V | | 4-sigma | 99.9937% | 3.2995V to 3.3243V | | 6-sigma | 99.99966% | 3.2933V to 3.3305V | ## Step 4: Add Marginal Limits (Warning Zone) Marginal limits create a warning zone between the soft limit and the hard fail. A board in the marginal zone passes but gets flagged for monitoring. | Zone | Range | Result | |------|-------|--------| | Pass | mean +/- 2-sigma | Normal pass | | Marginal | mean +/- 2-sigma to mean +/- 3-sigma | Pass with warning | | Fail | outside mean +/- 3-sigma | Hard fail | ## Step 5: Configure Limits in OpenHTF with TofuPilot ```python filename="vdd_test.py" import openhtf as htf from openhtf.util import units from tofupilot.openhtf import TofuPilot # Derived from 40 production samples: mean=3.3119V, std=0.0031V VDD_HARD_LOW = 3.3026 # mean - 3-sigma VDD_HARD_HIGH = 3.3212 # mean + 3-sigma VDD_MARGINAL_LOW = 3.3057 # mean - 2-sigma VDD_MARGINAL_HIGH = 3.3181 # mean + 2-sigma @htf.measures( htf.Measurement("vdd_rail_voltage") .in_range( minimum=VDD_HARD_LOW, maximum=VDD_HARD_HIGH, marginal_minimum=VDD_MARGINAL_LOW, marginal_maximum=VDD_MARGINAL_HIGH, ) .with_units(units.VOLT) .doc("3.3V rail with marginal zone [2-sigma, 3-sigma] for drift monitoring.") ) def measure_vdd(test): voltage = read_voltage_at_tp12() test.measurements.vdd_rail_voltage = voltage def main(): test = htf.Test( measure_vdd, test_name="VDD Rail Validation", ) with TofuPilot(test): test.execute(test_start=lambda: "SN-001") if __name__ == "__main__": main() ``` OpenHTF marginal outcome maps to `MARGINAL_PASS` in TofuPilot. You can filter by this outcome in the dashboard to track drift trends without stopping the line. ## Step 6: Track Limit Drift in TofuPilot Re-run the sigma analysis monthly and compare: ```python filename="refresh_limits.py" import numpy as np from datetime import datetime def compute_sigma_limits(samples: list[float], k: float = 3.0) -> dict: arr = np.array(samples) mean = np.mean(arr) std = np.std(arr, ddof=1) return { "n": len(arr), "mean": round(mean, 6), "std": round(std, 6), "low": round(mean - k * std, 6), "high": round(mean + k * std, 6), "sigma": k, "computed_at": datetime.utcnow().isoformat(), } ``` Compare new limits to current production limits before deploying. A shift in mean greater than 1-sigma warrants a process investigation. | Review Trigger | Action | |----------------|--------| | Mean shift > 1-sigma | Investigate root cause before updating | | Std increase > 20% | Check component reel change or station calibration | | Marginal rate > 5% | Tighten limits or improve process | | New sample N > 500 | Re-evaluate sigma level (consider 4-sigma) | ### Build a Test Data Strategy URL: https://www.tofupilot.com/guides/how-to-build-a-test-data-strategy-with-tofupilot Define what test data to capture, how to name procedures and measurements, and how to organize everything in TofuPilot for long-term traceability. A test data strategy decides what you capture, how you name it, and how you organize it before you write your first test. Getting this right early saves months of cleanup later. Getting it wrong means your analytics are noisy, your traceability has gaps, and your team wastes time decoding inconsistent data. ## What Data to Capture Every test run should include four categories of data: ### Measurements with Limits These are the quantitative values that determine pass/fail. Always define limits in code, not in post-processing. This ensures every run is evaluated against the same criteria. | Data Type | Example | Why It Matters | |-----------|---------|---------------| | Parametric measurements | Voltage, current, resistance | Enables Cpk analysis and trend detection | | Functional checks | Communication response, boot time | Validates system-level behavior | | Environmental readings | Temperature, humidity during test | Explains measurement variation | ### Unit Identity Every unit needs a unique serial number. If your product has sub-assemblies, track those serial numbers too. This is what lets you trace a field failure back through every test it ever went through. ### Metadata Context that doesn't have limits but matters for analysis: firmware version, hardware revision, operator ID, test station name. When you're investigating why yield dropped on Tuesday, metadata is how you find the cause. ### Attachments Log files, waveform captures, images from optical inspection. Not every run needs attachments, but when a failure investigation starts, you'll want them. ## Naming Conventions Consistent naming is the difference between data you can query and data you have to dig through manually. ### Procedure Names Use a clear hierarchy: `{product}_{stage}_{test_type}`. Keep names lowercase with underscores. | Pattern | Example | |---------|---------| | `{product}_{stage}_{test}` | `sensor_v2_evt_functional` | | `{product}_{stage}_{test}` | `motor_ctrl_pvt_burn_in` | | `{product}_{stage}_{test}` | `power_supply_dvt_thermal` | Don't embed dates or station IDs in procedure names. TofuPilot tracks those as separate fields. ### Measurement Names Use `{component}_{parameter}` format. Be specific enough that someone unfamiliar with the test can understand what was measured. ```python filename="test_naming_example.py" # Well-named measurements for a power supply test import openhtf as htf from openhtf.util import units from tofupilot.openhtf import TofuPilot @htf.measures( # Good: specific component, clear parameter htf.Measurement("rail_3v3_voltage") .in_range(minimum=3.25, maximum=3.35) .with_units(units.VOLT), htf.Measurement("rail_3v3_ripple") .in_range(maximum=30) .with_units(units.MILLIVOLT), htf.Measurement("rail_5v_voltage") .in_range(minimum=4.9, maximum=5.1) .with_units(units.VOLT), htf.Measurement("rail_5v_load_regulation_pct") .in_range(maximum=2.0), htf.Measurement("input_current_idle") .in_range(maximum=0.050) .with_units(units.AMPERE), htf.Measurement("thermal_shutdown_temp") .in_range(minimum=145, maximum=155) .with_units(units.DEGREE_CELSIUS), ) def power_supply_validation(test): test.measurements.rail_3v3_voltage = 3.301 test.measurements.rail_3v3_ripple = 18.4 test.measurements.rail_5v_voltage = 5.03 test.measurements.rail_5v_load_regulation_pct = 1.2 test.measurements.input_current_idle = 0.0321 test.measurements.thermal_shutdown_temp = 150.2 def main(): test = htf.Test(power_supply_validation) with TofuPilot(test): test.execute(test_start=lambda: "PSU-2024-0891") if __name__ == "__main__": main() ``` Avoid these naming mistakes: | Bad Name | Problem | Better Name | |----------|---------|-------------| | `test1` | Meaningless | `rail_3v3_voltage` | | `voltage` | Which voltage? | `rail_5v_voltage` | | `V_out_3.3V_meas` | Inconsistent casing and format | `rail_3v3_output_voltage` | | `temperature` | Which temperature, what unit? | `thermal_shutdown_temp` | ### Serial Number Format Pick a format and enforce it. Common patterns: | Format | Example | Use Case | |--------|---------|----------| | `{PREFIX}-{YEAR}-{SEQ}` | `PCB-2024-00042` | General manufacturing | | `{PRODUCT}-{LOT}-{SEQ}` | `SNS-L0287-015` | Lot-based production | | `{SITE}-{LINE}-{DATE}-{SEQ}` | `SH-A3-240315-0001` | Multi-site tracking | ## Organizing Procedures by Production Phase Structure your procedures to match your actual production flow. Each phase of testing should be a separate procedure in TofuPilot. ``` EVT (Engineering Validation) ├── sensor_v2_evt_power_on ├── sensor_v2_evt_functional ├── sensor_v2_evt_environmental └── sensor_v2_evt_reliability DVT (Design Validation) ├── sensor_v2_dvt_incoming_inspection ├── sensor_v2_dvt_calibration ├── sensor_v2_dvt_functional └── sensor_v2_dvt_burn_in PVT (Production Validation) ├── sensor_v2_pvt_smt_inspection ├── sensor_v2_pvt_ict ├── sensor_v2_pvt_functional └── sensor_v2_pvt_final_qc ``` This structure gives you per-phase FPY, lets you compare yield between EVT and PVT, and makes it obvious which test caught a failure. ## Adding Metadata with OpenHTF Use TofuPilot parameters to attach context that isn't a measurement: ```python filename="test_with_metadata.py" # Attaching sub-unit serial numbers for BOM traceability import openhtf as htf from tofupilot.openhtf import TofuPilot @htf.measures( htf.Measurement("boot_time_ms").in_range(maximum=500), htf.Measurement("self_test_result").equals("PASS"), ) def functional_check(test): test.measurements.boot_time_ms = 230 test.measurements.self_test_result = "PASS" def main(): test = htf.Test(functional_check) with TofuPilot( test, procedure_id="sensor_v2_pvt_functional", sub_units=[ {"serial_number": "WIFI-MOD-2024-0331"}, {"serial_number": "BT-MOD-2024-0887"}, ], ): test.execute(test_start=lambda: "SENSOR-2024-2210") if __name__ == "__main__": main() ``` Sub-unit serial numbers let you trace which specific components went into each assembly. When a component lot has issues, you can query TofuPilot to find every finished unit that contains affected parts. ## Data Retention Checklist Before you start collecting data, answer these questions: 1. **What compliance standards apply?** ISO 13485 (medical), AS9100 (aerospace), and IATF 16949 (automotive) all have specific data retention requirements. 2. **How long do you need to keep data?** Product lifetime plus warranty period is a common baseline. Medical devices often require 15+ years. 3. **What data needs to be immutable?** Test results used for regulatory compliance should never be editable after the fact. 4. **Who needs access?** Define roles early. Test engineers, quality managers, and customers may need different views of the same data. TofuPilot stores all test data with full audit history. Every run is timestamped and linked to its procedure, station, and operator. You don't need to build your own retention system. ### EMC Pre-Compliance Testing: A Complete Guide URL: https://www.tofupilot.com/guides/emc-pre-compliance-testing-a-complete-guide Learn how to set up EMC pre-compliance tests, log radiated and conducted emissions data, and track EMC margins with TofuPilot. # EMC Pre-Compliance Testing: A Complete Guide Failing EMC certification costs weeks of delay and tens of thousands in retest fees. Pre-compliance testing in your own lab catches problems early, when they're cheap to fix. TofuPilot tracks your EMC test data so you can monitor margins, compare design revisions, and build confidence before the formal test. ## What EMC Testing Covers | Test | Standard | What it measures | |------|----------|-----------------| | Radiated emissions | CISPR 32, FCC Part 15 | RF energy radiated from the product | | Conducted emissions | CISPR 32, FCC Part 15 | RF noise on power lines | | Radiated immunity | IEC 61000-4-3 | Resistance to external RF fields | | ESD immunity | IEC 61000-4-2 | Resistance to electrostatic discharge | | Surge immunity | IEC 61000-4-5 | Resistance to power line surges | | Conducted immunity | IEC 61000-4-6 | Resistance to RF on cables | Pre-compliance focuses on emissions (radiated and conducted) because they're the most common failure modes and can be tested with relatively affordable equipment. ## Pre-Compliance Equipment | Equipment | Purpose | Typical cost | |-----------|---------|-------------| | Near-field probe set | Locate emission sources on PCB | $500-2,000 | | Spectrum analyzer | Measure frequency content | $5,000-30,000 | | LISN (Line Impedance Stabilization Network) | Conducted emissions measurement | $2,000-5,000 | | EMC antenna (biconical + log-periodic) | Radiated emissions measurement | $3,000-8,000 | | Shielded enclosure or open area test site | Controlled measurement environment | $10,000+ | You don't need a fully equipped chamber for pre-compliance. A spectrum analyzer with near-field probes catches most issues at the board level. ## Logging EMC Data to TofuPilot ### Radiated Emissions Scan ```python filename="emc_radiated_test.py" from tofupilot import TofuPilotClient client = TofuPilotClient() # Peak emissions at key frequencies emissions = [ {"freq_mhz": 30, "level_dbuv_m": 28.5, "limit_dbuv_m": 40.0}, {"freq_mhz": 100, "level_dbuv_m": 32.1, "limit_dbuv_m": 43.5}, {"freq_mhz": 230, "level_dbuv_m": 35.8, "limit_dbuv_m": 46.0}, {"freq_mhz": 500, "level_dbuv_m": 22.3, "limit_dbuv_m": 46.0}, {"freq_mhz": 1000, "level_dbuv_m": 18.7, "limit_dbuv_m": 50.0}, ] measurements = [] all_pass = True for e in emissions: margin = e["limit_dbuv_m"] - e["level_dbuv_m"] passed = margin > 0 if not passed: all_pass = False measurements.append({ "name": f"radiated_{e['freq_mhz']}mhz", "value": e["level_dbuv_m"], "unit": "dBuV/m", "limit_high": e["limit_dbuv_m"], }) measurements.append({ "name": f"margin_{e['freq_mhz']}mhz", "value": margin, "unit": "dB", "limit_low": 6.0, # 6dB margin target }) client.create_run( procedure_id="EMC-RADIATED-PRECOMPLIANCE", unit_under_test={ "serial_number": "PROTO-REV-C", "part_number": "PRODUCT-V2", }, run_passed=all_pass, steps=[{ "name": "Radiated Emissions 30MHz-1GHz", "step_type": "measurement", "status": all_pass, "measurements": measurements, }], ) ``` ### Conducted Emissions ```python filename="emc_conducted_test.py" # Conducted emissions on power input lines conducted = [ {"freq_mhz": 0.15, "level_dbuv": 52.3, "limit_avg": 56.0, "limit_qp": 66.0}, {"freq_mhz": 0.5, "level_dbuv": 48.7, "limit_avg": 46.0, "limit_qp": 56.0}, {"freq_mhz": 5.0, "level_dbuv": 38.2, "limit_avg": 46.0, "limit_qp": 56.0}, {"freq_mhz": 30.0, "level_dbuv": 32.1, "limit_avg": 46.0, "limit_qp": 56.0}, ] measurements = [] for c in conducted: measurements.append({ "name": f"conducted_avg_{c['freq_mhz']}mhz", "value": c["level_dbuv"], "unit": "dBuV", "limit_high": c["limit_avg"], }) client.create_run( procedure_id="EMC-CONDUCTED-PRECOMPLIANCE", unit_under_test={"serial_number": "PROTO-REV-C"}, run_passed=True, steps=[{ "name": "Conducted Emissions 150kHz-30MHz", "step_type": "measurement", "status": True, "measurements": measurements, }], ) ``` ## Tracking EMC Margins The goal of pre-compliance isn't just pass/fail. It's knowing your margin. A product that passes with 2dB of margin will likely fail at the test house (measurement uncertainty alone is 3-5dB). | Margin | Risk level | Action | |--------|-----------|--------| | > 10 dB | Low | Proceed to certification | | 6-10 dB | Medium | Proceed, but have mitigation plans ready | | 3-6 dB | High | Fix before certification | | < 3 dB | Very high | Will likely fail certification | Track margins in TofuPilot across design revisions. If Rev A had 4dB margin at 230MHz and Rev B has 8dB after adding a filter, you can see the improvement quantitatively. ## Comparing Design Revisions EMC performance changes with every board revision, layout change, and component swap. TofuPilot lets you compare emissions data across revisions side by side. ``` Rev A (no filter): Rev B (added pi filter): ┌─────────────────────┐ ┌─────────────────────┐ │ 230MHz: 42.1 dBuV/m │ │ 230MHz: 35.8 dBuV/m │ │ Limit: 46.0 dBuV/m │ │ Limit: 46.0 dBuV/m │ │ Margin: 3.9 dB ⚠ │ │ Margin: 10.2 dB ✓ │ └─────────────────────┘ └─────────────────────┘ ``` The pi filter bought 6.3dB of margin at 230MHz. This kind of data-driven design feedback is only possible when you track every test iteration. ## Common EMC Failure Modes | Failure | Typical cause | Fix | |---------|--------------|-----| | Harmonics of switching frequency | DC-DC converter, clock oscillator | Add input/output filtering, spread-spectrum clocking | | Cable-radiated emissions | Unshielded cables acting as antennas | Add common-mode chokes, use shielded cables | | High-frequency broadband | Fast digital edges, poor grounding | Slow edge rates, improve ground plane | | Low-frequency conducted | Switching power supply ripple | Add differential-mode filter on input | Track which failure modes appear across products in TofuPilot. If every product fails at the 3rd harmonic of the switching frequency, standardize your input filter design. ## ESD Testing ESD is the most common immunity test failure. Log ESD test results systematically. ```python filename="esd_test.py" esd_levels = [ {"mode": "contact", "voltage_kv": 4, "result": "pass"}, {"mode": "contact", "voltage_kv": 6, "result": "pass"}, {"mode": "contact", "voltage_kv": 8, "result": "fail_recoverable"}, {"mode": "air", "voltage_kv": 8, "result": "pass"}, {"mode": "air", "voltage_kv": 15, "result": "pass"}, ] measurements = [] for test in esd_levels: passed = test["result"] == "pass" measurements.append({ "name": f"esd_{test['mode']}_{test['voltage_kv']}kv", "value": 1 if passed else 0, "unit": "pass/fail", "limit_low": 1, }) client.create_run( procedure_id="EMC-ESD-PRECOMPLIANCE", unit_under_test={"serial_number": "PROTO-REV-C"}, run_passed=all(t["result"] == "pass" for t in esd_levels), steps=[{ "name": "ESD Immunity per IEC 61000-4-2", "step_type": "measurement", "status": True, "measurements": measurements, }], ) ``` ## From Pre-Compliance to Certification Pre-compliance data in TofuPilot serves as your engineering notebook for EMC. When you arrive at the test house: - You know your margins at every frequency - You know which design changes improved or degraded performance - You have a history of every test iteration - If you fail, you have data to guide the fix instead of starting from scratch ### Automated Verification and Validation URL: https://www.tofupilot.com/guides/automated-verification-and-validation-with-tofupilot Learn how to automate hardware verification and validation workflows using TofuPilot for structured test evidence and traceability. # Automated Verification and Validation with TofuPilot Verification and validation (V&V) in hardware is still largely manual: engineers run tests, record results in documents, and someone reviews the evidence. TofuPilot automates the data capture side so engineers can focus on the analysis instead of the paperwork. ## V&V in Hardware Development Verification answers: "Did we build the product right?" (Does it meet its specifications?) Validation answers: "Did we build the right product?" (Does it meet user needs?) Both require structured test evidence. In regulated industries (medical devices, aerospace, automotive), this evidence must be traceable, timestamped, and immutable. | Phase | Purpose | Typical tests | Evidence needed | |-------|---------|--------------|----------------| | EVT | Engineering validation | Functional, environmental, reliability | Test reports, measurement data | | DVT | Design verification | Full spec compliance, regulatory | Formal test records, traceability | | PVT | Production validation | Manufacturing process qualification | Statistical data, Cpk analysis | | Production | Ongoing verification | In-line test, final test | Per-unit records, yield data | ## The Manual V&V Problem Traditional V&V workflows look like this: 1. Engineer writes a test protocol (Word document) 2. Technician runs the tests manually or semi-automatically 3. Results are recorded in a spreadsheet or paper form 4. Engineer reviews results and writes a test report 5. Quality reviews and approves the report 6. Documents are filed in a QMS This process is slow, error-prone, and disconnected from the actual test data. The spreadsheet is a copy of the data, not the data itself. If someone transcribes a number wrong, the V&V record is wrong. ## Automating V&V with TofuPilot ### Step 1: Define Your Verification Test Procedures Create a procedure in TofuPilot for each verification test in your protocol. ```python filename="evt_thermal.py" from tofupilot import TofuPilotClient client = TofuPilotClient() # EVT thermal cycling test client.create_run( procedure_id="EVT-THERMAL-CYCLE", unit_under_test={ "serial_number": "EVT-PROTO-003", "part_number": "PRODUCT-V2.1", }, run_passed=True, steps=[{ "name": "Post-Cycle Functional Check", "step_type": "measurement", "status": True, "measurements": [ { "name": "output_voltage", "value": 5.02, "unit": "V", "limit_low": 4.90, "limit_high": 5.10, }, { "name": "output_ripple_mv", "value": 12.3, "unit": "mV", "limit_high": 50.0, }, { "name": "efficiency_pct", "value": 91.2, "unit": "%", "limit_low": 88.0, }, ], }, { "name": "Thermal Performance", "step_type": "measurement", "status": True, "measurements": [ { "name": "junction_temp_max", "value": 82.4, "unit": "°C", "limit_high": 105.0, }, { "name": "thermal_resistance", "value": 3.8, "unit": "°C/W", "limit_high": 5.0, }, ], }], ) ``` ### Step 2: Map Requirements to Measurements Each measurement in TofuPilot corresponds to a requirement in your spec. Use consistent naming to create a clear mapping. | Requirement ID | Requirement | TofuPilot Measurement | Limit | |---------------|-------------|----------------------|-------| | REQ-ELEC-001 | Output voltage: 5V +/- 2% | `output_voltage` | 4.90-5.10 V | | REQ-ELEC-002 | Ripple: < 50mV | `output_ripple_mv` | < 50 mV | | REQ-ELEC-003 | Efficiency: > 88% | `efficiency_pct` | > 88% | | REQ-THERM-001 | Junction temp: < 105C | `junction_temp_max` | < 105 C | This mapping turns your test data into verification evidence. Every measurement in TofuPilot is a timestamped, immutable record that a requirement was tested. ### Step 3: Run Tests Across V&V Phases Use the same procedures across EVT, DVT, and PVT. TofuPilot stores all results, so you can compare performance across phases. ``` EVT (3 prototypes): DVT (30 units): PVT (300 units): ├── EVT-THERMAL ├── DVT-THERMAL ├── PVT-THERMAL ├── EVT-VIBRATION ├── DVT-VIBRATION ├── PVT-PROCESS-QUAL ├── EVT-FUNCTIONAL ├── DVT-EMC └── PVT-YIELD-ANALYSIS └── EVT-RELIABILITY ├── DVT-SAFETY └── DVT-FUNCTIONAL ``` ### Step 4: Generate V&V Evidence TofuPilot's stored test data serves as your verification evidence: - **Per-unit test records**: Every measurement for every unit, with timestamps and pass/fail status - **Statistical summaries**: Cpk, mean, standard deviation, distribution across the test population - **Traceability**: Link from requirement to test procedure to test result to specific unit - **Trend data**: How measurements changed across EVT, DVT, and PVT phases ## Traceability for Audits When an auditor asks "Show me the evidence that REQ-ELEC-001 was verified," you can: 1. Open TofuPilot 2. Filter by procedure (e.g., `DVT-FUNCTIONAL`) 3. Show all `output_voltage` measurements across the DVT population 4. Show the pass/fail status, limits, and statistical summary No digging through binders. No searching shared drives. The data is structured, searchable, and timestamped. ## Comparing V&V Across Design Revisions When you make a design change between EVT and DVT, TofuPilot lets you compare the same measurements across phases. Did the thermal performance improve with the new heatsink design? Compare `junction_temp_max` distributions between EVT and DVT. The data answers the question objectively. ## Continuous Verification in Production V&V doesn't end at PVT. Every unit tested in production is a data point that verifies the design continues to meet its specifications. TofuPilot's production dashboards show ongoing verification: - Are all measurements still within limits? - Is the process capability (Cpk) maintained? - Are there any new failure modes that weren't seen during V&V? If a measurement that was comfortably within limits during DVT starts approaching a limit in production, you know the process or components have changed. Investigate before it becomes a field issue. ### What Is FMEA for Manufacturing Test URL: https://www.tofupilot.com/guides/what-is-fmea-for-manufacturing-test-with-tofupilot FMEA identifies potential failure modes before they happen. Learn how to apply FMEA to manufacturing test processes and track corrective actions with TofuPilot. # What Is FMEA for Manufacturing Test with TofuPilot Failure Mode and Effects Analysis (FMEA) is a structured method for identifying what can go wrong, how severe the consequences are, and what controls exist to catch it. Applied to manufacturing test, FMEA helps you decide which test steps matter most and where to invest in detection. This guide covers how FMEA works, the difference between DFMEA and PFMEA, and how TofuPilot data feeds your FMEA process. ## How FMEA Works FMEA scores each potential failure mode on three dimensions: | Factor | What It Measures | Scale | |--------|-----------------|-------| | Severity (S) | How bad is it if this failure reaches the customer? | 1-10 | | Occurrence (O) | How often does this failure mode happen? | 1-10 | | Detection (D) | How likely is your current test to catch it? | 1-10 | The Risk Priority Number (RPN) is S x O x D. Higher RPN means higher priority. A failure that's severe, frequent, and hard to detect gets the highest score. ### Example FMEA for a Power Supply | Failure Mode | Effect | S | O | D | RPN | Action | |-------------|--------|---|---|---|-----|--------| | Solder bridge on output FET | Short circuit, potential fire | 10 | 3 | 2 | 60 | AOI + hipot test covers this | | Wrong capacitor value | Output ripple out of spec | 6 | 2 | 4 | 48 | Add ripple measurement to FCT | | Cold solder on connector | Intermittent connection in field | 8 | 4 | 7 | 224 | Add pull test or thermal cycling screen | | Firmware flash failure | Unit DOA | 9 | 2 | 1 | 18 | Self-test catches this | The cold solder on the connector has the highest RPN because it's hard to detect (D=7). That's where you invest in better testing. ## DFMEA vs PFMEA | Type | Scope | Who Leads | When | |------|-------|-----------|------| | DFMEA (Design FMEA) | Product design failures | Design engineering | EVT/DVT | | PFMEA (Process FMEA) | Manufacturing process failures | Process/test engineering | PVT/production | DFMEA asks: what can fail in the design? PFMEA asks: what can go wrong during manufacturing? For test engineering, PFMEA is where you define which tests are needed and why. ## How TofuPilot Data Feeds FMEA FMEA requires real data for the Occurrence and Detection scores. Guessing these numbers makes the analysis unreliable. TofuPilot provides the data you need: | FMEA Input | TofuPilot Source | |-----------|-----------------| | Occurrence rate | Failure Pareto by test step | | Detection effectiveness | Compare production test yield to field return rate | | Failure mode distribution | Measurement distributions showing which parameters drift | | Trend data | Control charts showing whether a failure mode is increasing | ## Prerequisites - Python 3.10+ - OpenHTF installed (`pip install openhtf`) - TofuPilot Python SDK installed (`pip install tofupilot`) ## Step 1: Map FMEA Actions to Test Phases Each FMEA action item that requires a test becomes an OpenHTF phase. Include the FMEA reference in the phase docstring for traceability. ```python filename="fmea_driven_test.py" import openhtf as htf from openhtf.util import units @htf.measures( htf.Measurement("output_ripple_mV") .in_range(maximum=50) .with_units(units.MILLIVOLT), ) def phase_ripple_check(test): """PFMEA item #2: Detect wrong capacitor via ripple measurement.""" test.measurements.output_ripple_mV = 28.5 @htf.measures( htf.Measurement("connector_resistance_mOhm") .in_range(maximum=100) .with_units(units.OHM), ) def phase_connector_check(test): """PFMEA item #3: Detect cold solder on output connector.""" test.measurements.connector_resistance_mOhm = 42.0 @htf.measures( htf.Measurement("hipot_result").equals("PASS"), ) def phase_hipot(test): """PFMEA item #1: Verify no shorts on output stage.""" test.measurements.hipot_result = "PASS" ``` ## Step 2: Log Results and Update FMEA Connect the test to TofuPilot. As production data accumulates, update your FMEA occurrence and detection scores with real numbers. ```python filename="fmea_driven_test.py" from tofupilot.openhtf import TofuPilot test = htf.Test( phase_hipot, phase_ripple_check, phase_connector_check, ) with TofuPilot(test): test.execute(test_start=lambda: input("Scan serial: ")) ``` ## Step 3: Review and Iterate TofuPilot tracks results per test step. Open the Analytics tab to review: - **Failure Pareto** shows which failure modes actually occur most often (updates your O score) - **Measurement distributions** show whether your limits catch marginal units (validates your D score) - **Yield trends** show whether corrective actions are working (RPN should decrease over time) FMEA is a living document. Review it quarterly using TofuPilot data to update scores and reprioritize actions. A failure mode that was rare six months ago might be common now due to a supplier change. ### CI/CD for Hardware Testing with TofuPilot URL: https://www.tofupilot.com/guides/cicd-for-hardware-testing-with-tofupilot Learn how to integrate hardware test automation into CI/CD pipelines using TofuPilot for continuous validation and gating. # CI/CD for Hardware Testing with TofuPilot Software teams have had CI/CD for years. Hardware teams still gate releases with spreadsheets and manual sign-offs. TofuPilot bridges that gap by treating hardware test results as pipeline artifacts, so you can automate validation the same way you automate builds. ## Why CI/CD Matters for Hardware Hardware test data is the equivalent of a software test suite. Every unit that comes off the line runs through functional tests, calibration checks, and environmental screens. The question isn't whether you have tests. It's whether you can automatically block a shipment when test data says "stop." Traditional hardware workflows rely on someone reviewing a report. CI/CD for hardware means the pipeline reviews the data and makes the call. ## Architecture ``` ┌──────────────┐ ┌──────────────┐ ┌──────────────┐ │ Test Station │────▶│ TofuPilot │────▶│ CI/CD Gate │ │ (OpenHTF / │ │ (Results │ │ (Pass/Fail │ │ pytest) │ │ + API) │ │ Decision) │ └──────────────┘ └──────────────┘ └──────────────┘ ``` Test stations push results to TofuPilot. Your CI/CD system queries TofuPilot's API to check whether a batch meets release criteria. ## Setting Up Continuous Hardware Validation ### Step 1: Push Results from Test Stations Every test run uploads to TofuPilot automatically. If you're using OpenHTF, add the TofuPilot output callback. If you're using pytest or a custom framework, use the Python client. ```python filename="station_test.py" from tofupilot import TofuPilotClient client = TofuPilotClient() client.create_run( procedure_id="EVT-POWER-CYCLE", unit_under_test={"serial_number": "UNIT-4521"}, run_passed=True, steps=[{ "name": "Power Rail Check", "step_type": "measurement", "status": True, "measurements": [{ "name": "vcc_3v3", "value": 3.31, "unit": "V", "limit_low": 3.25, "limit_high": 3.35, }], }], ) ``` ### Step 2: Query Results in Your Pipeline Use TofuPilot's API to check batch-level pass rates before allowing a release to proceed. ```python filename="ci_gate.py" # CI/CD gate script: check batch FPY before release import requests import sys API_KEY = os.environ["TOFUPILOT_API_KEY"] PROCEDURE = "EVT-POWER-CYCLE" MIN_FPY = 0.95 MIN_UNITS = 50 response = requests.get( "https://app.tofupilot.com/api/v1/runs", headers={"Authorization": f"Bearer {API_KEY}"}, params={"procedure_id": PROCEDURE, "limit": 200}, ) runs = response.json() passed = sum(1 for r in runs if r["run_passed"]) total = len(runs) fpy = passed / total if total > 0 else 0 print(f"FPY: {fpy:.1%} ({passed}/{total} units)") if total < MIN_UNITS: print(f"Not enough units tested ({total} < {MIN_UNITS})") sys.exit(1) if fpy < MIN_FPY: print(f"FPY below threshold ({fpy:.1%} < {MIN_FPY:.0%})") sys.exit(1) print("Gate passed. Batch approved for release.") ``` ### Step 3: Integrate with GitHub Actions or GitLab CI ```yaml filename=".github/workflows/hardware-gate.yml" name: Hardware Release Gate on: workflow_dispatch: inputs: batch_id: description: "Batch to validate" required: true jobs: validate: runs-on: ubuntu-latest steps: - uses: actions/checkout@v4 - uses: actions/setup-python@v5 with: python-version: "3.11" - run: pip install requests - run: python ci_gate.py env: TOFUPILOT_API_KEY: ${{ secrets.TOFUPILOT_API_KEY }} ``` ## Gating Strategies | Strategy | When to use | How it works | |----------|-------------|--------------| | FPY threshold | Production release | Block if first-pass yield drops below target | | Zero critical failures | Safety-critical products | Block if any critical measurement fails | | Cpk minimum | Process validation | Block if process capability index is too low | | Trend detection | Early warning | Alert if yield is declining over last N batches | ## Test Like You Fly The aerospace principle of "test like you fly" means production tests should mirror real operating conditions. CI/CD for hardware makes this practical by automating the feedback loop: 1. Run the same tests in EVT, DVT, and PVT 2. Compare results across phases in TofuPilot 3. Gate each phase transition on measured data, not opinions When your DVT thermal cycling shows a 3% yield drop compared to EVT, TofuPilot's comparison dashboards surface it immediately. No waiting for someone to pull a report. ## What This Replaces Most hardware teams gate releases with a combination of Excel reports, email chains, and meeting sign-offs. CI/CD for hardware doesn't eliminate human judgment. It eliminates the manual data gathering that delays judgment. Engineers spend time analyzing instead of aggregating. ### Calculate True First Pass Yield URL: https://www.tofupilot.com/guides/how-to-calculate-true-first-pass-yield-with-tofupilot Learn the FPY formula, how it differs from rework yield and rolled throughput yield, and how TofuPilot tracks it automatically. First pass yield (FPY) is the percentage of units that pass testing on their first attempt, without rework or retesting. It's the single most important metric for understanding your production quality, and most teams measure it wrong. ## The FPY Formula FPY = (units passing on first test) / (total unique units tested) If you test 200 boards today and 180 pass on the first attempt, your FPY is 90%. The 20 that failed, even if they pass after rework and retest, don't count toward FPY. This distinction matters. A line reporting 98% "yield" might actually have 85% FPY with the rest passing only after rework. Those numbers tell very different stories about process health. ## FPY vs. Other Yield Metrics Several yield metrics exist, and confusing them leads to bad decisions. | Metric | Formula | What it measures | |--------|---------|------------------| | **First pass yield (FPY)** | Pass on 1st attempt / Total units | Process capability at the test station | | **Rework yield** | Pass after rework / Total reworked | Effectiveness of your rework process | | **Output yield** | Total passing (including retests) / Total units | Final throughput, hides rework cost | | **Rolled throughput yield (RTY)** | FPY₁ x FPY₂ x ... x FPYₙ | Probability a unit passes all stations without rework | Output yield is the number most teams report. It looks good on slides. But it hides the cost of rework, the strain on test capacity, and the risk of shipping marginal units. RTY is the most demanding metric. If you have three test stations each with 95% FPY, your RTY is 0.95 x 0.95 x 0.95 = 85.7%. That means only 86 out of 100 units flow through your entire line without touching rework. ## Writing Tests That Produce Accurate FPY Data FPY accuracy depends on your test scripts producing clean, structured results. Every run needs a serial number so TofuPilot can distinguish first attempts from retests of the same unit. ```python filename="board_functional_test.py" import openhtf as htf from openhtf.util import units from tofupilot.openhtf import TofuPilot @htf.measures( htf.Measurement("supply_voltage") .with_units(units.VOLT) .in_range(minimum=4.75, maximum=5.25), htf.Measurement("clock_frequency") .in_range(minimum=7.99, maximum=8.01), htf.Measurement("standby_current") .in_range(maximum=50), ) def test_power_and_clock(test): test.measurements.supply_voltage = 5.03 test.measurements.clock_frequency = 8.002 test.measurements.standby_current = 32 @htf.measures( htf.Measurement("comms_loopback_pass"), ) def test_communications(test): test.measurements.comms_loopback_pass = True def main(): test = htf.Test(test_power_and_clock, test_communications) with TofuPilot(test): test.execute(test_start=lambda: "SN-2026-00842") if __name__ == "__main__": main() ``` When this test runs, TofuPilot records the serial number, all measurements with their limits, and the pass/fail outcome. If `SN-2026-00842` fails and gets retested later, TofuPilot knows it's a retest, not a new unit. ## How TofuPilot Tracks FPY Automatically You don't need to compute FPY yourself. TofuPilot's analytics dashboard calculates it in real time from your test data. What the dashboard provides: - **FPY trend over time.** See daily, weekly, or monthly FPY for any product or station. Spot drops the day they happen, not weeks later in a quarterly review. - **FPY by station.** Compare stations running the same test. If Station 3 has 88% FPY while Station 1 and 2 are at 96%, the problem is the station, not the product. - **FPY by product revision.** Track whether a new board revision actually improved yield or made it worse. - **Unit history.** For any serial number, see every test attempt. Understand whether failures cluster on specific units or spread evenly. ## Common FPY Pitfalls **Counting retests as first attempts.** If your system doesn't track serial numbers, every test looks like a first attempt. Your FPY will be artificially low because failures and their retests both count as unique units. **Excluding known-bad units.** Some teams exclude units that "obviously" failed due to fixture issues or operator error. This inflates FPY and hides real problems. **Measuring too late.** FPY at final test captures problems from every upstream process. Measuring FPY at each station (and computing RTY) gives you much better isolation. **Setting limits too tight.** If your 3-sigma process capability doesn't fit inside your test limits, you'll see chronic low FPY even with a good process. Check your Cpk before blaming the line. ### What Is AOI, SPI, and AXI in PCB Manufacturing URL: https://www.tofupilot.com/guides/what-is-aoi-spi-and-axi-with-tofupilot AOI, SPI, and AXI are automated inspection methods for PCB manufacturing. Learn what each does, when to use them, and how to track inspection data. # What Is AOI, SPI, and AXI with TofuPilot AOI, SPI, and AXI are automated inspection methods used at different stages of PCB assembly. SPI checks solder paste before placement. AOI checks components and solder joints after reflow. AXI uses X-rays to inspect hidden connections. This guide covers what each method does, where it fits in the SMT line, and how to track inspection results with TofuPilot. ## Where Each Method Fits A typical SMT line runs these inspections in order: | Step | Process | Inspection | |------|---------|-----------| | 1 | Solder paste printing | SPI (Solder Paste Inspection) | | 2 | Component placement | None (pick-and-place is self-checking) | | 3 | Reflow soldering | AOI (Automated Optical Inspection) | | 4 | Hidden joints (BGA, QFN) | AXI (Automated X-Ray Inspection) | | 5 | Through-hole/wave solder | AOI (second pass) | ## SPI: Solder Paste Inspection SPI measures the volume, height, area, and position of solder paste deposits after printing. It runs before any components are placed. | What SPI Measures | Why It Matters | |-------------------|---------------| | Paste volume | Too little causes opens, too much causes bridges | | Paste height | Indicates stencil wear or pressure issues | | Paste area | Detects misalignment or clogged apertures | | Paste offset | Catches stencil registration errors | SPI catches problems at the cheapest point in the process. Fixing a paste defect costs pennies. Finding the same defect after reflow costs dollars. Finding it in the field costs hundreds. ## AOI: Automated Optical Inspection AOI uses cameras (often with angled lighting or 3D measurement) to inspect PCBs after soldering. It checks for: | Defect Type | What AOI Sees | |-------------|--------------| | Missing component | Empty pad where a part should be | | Tombstoned component | Part standing on one end | | Wrong polarity | Marking orientation incorrect | | Solder bridge | Shorts between adjacent pads | | Insufficient solder | Incomplete fillet | | Wrong component | Size or marking mismatch | AOI is fast (seconds per board) and non-contact. It's the workhorse inspection method for surface-mount assemblies. ### AOI Limitations AOI cannot see: - Solder joints hidden under packages (BGA, QFN, LGA) - Cold solder joints that look correct visually - Internal component defects That's where AXI comes in. ## AXI: Automated X-Ray Inspection AXI uses X-ray imaging to inspect solder joints that are hidden from optical inspection. It's essential for: | Package Type | Why AXI Is Needed | |-------------|------------------| | BGA | Solder balls are underneath the package | | QFN | Thermal pad and ground connections are hidden | | LGA | All connections are under the package | | Connectors | Press-fit or hidden solder joints | AXI detects voids, head-in-pillow defects, bridging under packages, and incomplete reflow. It's slower and more expensive than AOI, so it's typically used on boards with BGA or QFN components rather than every board. ## Choosing the Right Inspection | Question | Answer | |----------|--------| | Do we have BGA or QFN? | Use AXI | | Surface-mount only, no hidden joints? | AOI is sufficient | | High solder defect rate? | Add SPI to catch paste issues early | | Low volume, no AOI machine? | Manual visual inspection with magnification | Most SMT lines use SPI + AOI as a minimum. Add AXI when the board design includes hidden solder joints. ## Tracking Inspection Results with TofuPilot Inspection machines output defect data per board. You can log this data to TofuPilot alongside your functional test results to see the full quality picture for each unit. ```python filename="inspection_log.py" import openhtf as htf @htf.measures( htf.Measurement("spi_result").equals("PASS"), htf.Measurement("spi_defect_count").in_range(maximum=0), ) def phase_spi_result(test): """Log SPI inspection result from paste printer.""" test.measurements.spi_result = "PASS" test.measurements.spi_defect_count = 0 @htf.measures( htf.Measurement("aoi_result").equals("PASS"), htf.Measurement("aoi_defect_count").in_range(maximum=0), ) def phase_aoi_result(test): """Log AOI inspection result from post-reflow inspection.""" test.measurements.aoi_result = "PASS" test.measurements.aoi_defect_count = 0 ``` ```python filename="inspection_log.py" from tofupilot.openhtf import TofuPilot test = htf.Test( phase_spi_result, phase_aoi_result, ) with TofuPilot(test): test.execute(test_start=lambda: input("Scan board serial: ")) ``` TofuPilot tracks inspection results per serial number. Open the Analytics tab to see defect rates by inspection type, defect Pareto charts, and trends across production lots. ### Build Full Unit Traceability URL: https://www.tofupilot.com/guides/how-to-build-full-unit-traceability-with-tofupilot Link every test result to a specific serial number and track each unit's complete test history through its production lifecycle. Unit traceability means linking every test result to a specific serial number so you can reconstruct the full history of any device. TofuPilot captures this automatically when you run OpenHTF tests with serial numbers, giving you a searchable unit history page for every DUT. ## Why Unit Traceability Matters When a field failure occurs, you need to answer: what tests did this unit pass, when, on which station, and with what measurements? Without traceability, you're digging through spreadsheets and log files. With it, you type a serial number and get the full picture. Regulatory frameworks like ISO 13485, AS9100, and IPC-1782 all require linking test records to individual units. TofuPilot stores this mapping automatically. ## Capturing Serial Numbers in OpenHTF OpenHTF uses a `test_start` trigger to capture the DUT serial number before any test phase runs. The simplest approach prompts the operator to scan or type the serial. ```python filename="test_traceability.py" # Production test with serial number capture and TofuPilot upload import openhtf as htf from openhtf.util import units from tofupilot.openhtf import TofuPilot @htf.measures( htf.Measurement("voltage_output").in_range(minimum=4.8, maximum=5.2).with_units(units.VOLT), htf.Measurement("current_draw").in_range(minimum=0.1, maximum=0.5).with_units(units.AMPERE), ) def power_test(test): test.measurements.voltage_output = 5.01 test.measurements.current_draw = 0.25 def main(): test = htf.Test(power_test) with TofuPilot(test): test.execute(test_start=lambda: input("Scan serial: ")) if __name__ == "__main__": main() ``` Every time this test runs, TofuPilot associates the measurements and pass/fail result with the scanned serial number. Run the same serial again and TofuPilot appends to its history. ## Programmatic Serial Number Input In automated lines, serial numbers come from barcode scanners or MES systems rather than operator input. You can pass the serial directly. ```python filename="test_automated_serial.py" # Automated serial input from barcode scanner import openhtf as htf from openhtf.util import units from tofupilot.openhtf import TofuPilot @htf.measures( htf.Measurement("resistance").in_range(minimum=95, maximum=105).with_units(units.OHM), ) def resistance_check(test): test.measurements.resistance = 100.2 def main(): # Serial comes from your barcode scanner or MES integration serial_number = read_barcode_scanner() test = htf.Test(resistance_check) with TofuPilot(test): test.execute(test_start=lambda: serial_number) if __name__ == "__main__": main() ``` ## Adding Unit Metadata You can attach metadata to test runs for clearer unit history organization by production stage. ```python filename="test_with_metadata.py" # Tag runs with production stage for clearer unit history import openhtf as htf from tofupilot.openhtf import TofuPilot @htf.measures( htf.Measurement("firmware_version"), htf.Measurement("boot_time").in_range(maximum=5.0), ) def firmware_check(test): test.measurements.firmware_version = "2.1.0" test.measurements.boot_time = 1.8 def main(): test = htf.Test(firmware_check) with TofuPilot(test): test.execute(test_start=lambda: "SN-20260312-001") if __name__ == "__main__": main() ``` ## Multi-Stage Unit History Real production involves multiple test stages: ICT, functional test, burn-in, final QA. Each stage runs as a separate OpenHTF test with the same serial number. TofuPilot stitches them together. ```python filename="test_multi_stage.py" # Each production stage uploads to TofuPilot with the same serial import openhtf as htf from tofupilot.openhtf import TofuPilot @htf.measures( htf.Measurement("ict_shorts").in_range(maximum=0), ) def ict_test(test): test.measurements.ict_shorts = 0 @htf.measures( htf.Measurement("functional_pass"), ) def functional_test(test): test.measurements.functional_pass = True def run_stage(phase, serial): test = htf.Test(phase) with TofuPilot(test): test.execute(test_start=lambda: serial) def main(): serial = "SN-20260312-001" run_stage(ict_test, serial) run_stage(functional_test, serial) if __name__ == "__main__": main() ``` Open the unit's page in TofuPilot's dashboard to see every test run, measurement, and pass/fail result for that serial number in chronological order. You can filter by procedure, station, or date range. ## Viewing Unit History in TofuPilot Once your tests upload results, go to TofuPilot's Units page and search by serial number. The unit detail page shows: - Every test run tied to that serial, across all procedures and stations - Individual measurements with pass/fail status and limits - Timestamps and station identifiers for each run - Links to related sub-assemblies (if configured) This is the audit trail that regulators and quality engineers need. No extra code required. ### What Is Zero-Defect Manufacturing URL: https://www.tofupilot.com/guides/what-is-zero-defect-manufacturing Zero-defect manufacturing aims to prevent defects rather than detect them. Learn what it means in practice, what it requires, and how test data supports it. # What Is Zero-Defect Manufacturing Zero-defect manufacturing (ZDM) is the principle that defects should be prevented, not just detected. Instead of accepting a defect rate and managing it, ZDM aims to eliminate defects at their source. This guide covers what zero-defect manufacturing means in practice, how it differs from traditional quality approaches, and how test data supports the goal. ## The Zero-Defect Principle Philip Crosby introduced the zero-defect concept in the 1960s. The core idea: the performance standard should be zero defects, not "an acceptable quality level." | Traditional Approach | Zero-Defect Approach | |---------------------|---------------------| | "Some defects are inevitable" | "Every defect has a preventable root cause" | | Accept AQL of 0.1-1.0% | Target 0% defect rate | | Focus on detection and sorting | Focus on prevention and process control | | Quality is a cost center | Quality is a savings generator | | Inspect quality in | Build quality in | Zero defects doesn't mean perfection is achieved on day one. It means perfection is the target, and every defect is treated as a process failure to be investigated and corrected, not as an acceptable cost of doing business. ## Zero Defects in Practice No production line truly has zero defects. The practical question is: how close can you get, and what does each step closer cost? | Defect Rate | FPY | DPMO | Sigma Level | Typical Industry | |------------|-----|------|-------------|-----------------| | 10% | 90% | 100,000 | 2.8 | Early production, new product | | 1% | 99% | 10,000 | 3.8 | Established consumer electronics | | 0.1% | 99.9% | 1,000 | 4.6 | Automotive components | | 0.01% | 99.99% | 100 | 5.2 | Medical devices, aerospace | | 0.00034% | 99.99966% | 3.4 | 6.0 | Six Sigma target | Moving from 99% to 99.9% FPY is harder than moving from 90% to 99%. Each order of magnitude requires deeper process understanding and more sophisticated detection. ## The Four Pillars of ZDM ### 1. Prevention Design the product and process so defects can't occur. | Prevention Method | Example | |------------------|---------| | Design for manufacturability (DFM) | Wider solder pads, larger component clearances | | Poka-yoke (mistake-proofing) | Connectors that only fit one way | | Process capability studies | Verify Cpk > 1.33 before production | | Supplier qualification | Qualify components and vendors before use | ### 2. Detection Catch defects as early as possible when prevention fails. | Detection Method | Stage | |-----------------|-------| | SPI, AOI, AXI | In-process inspection | | Functional test | After assembly | | EOL test | Before shipping | | ORT sampling | Ongoing reliability | ### 3. Prediction Use data to forecast defects before they occur. | Prediction Method | Data Source | |------------------|-----------| | SPC control charts | Process parameter trends | | Measurement drift detection | Test result trends | | Supplier quality tracking | IQC rejection rates | | ML-based prediction | Historical test data correlations | ### 4. Correction When defects occur, fix the root cause permanently. | Correction Method | Purpose | |------------------|---------| | Root cause analysis | Find the true cause, not the symptom | | CAPA | Document corrective and preventive actions | | Process change validation | Verify the fix works without introducing new defects | | Limit refinement | Tighten or add detection for the specific failure mode | ## Prerequisites - Python 3.10+ - OpenHTF installed (`pip install openhtf`) - TofuPilot Python SDK installed (`pip install tofupilot`) ## How Test Data Supports Zero Defects Structured test data is the foundation of all four pillars. Without data, prevention is guesswork, detection is incomplete, prediction is impossible, and correction is temporary. ```python filename="zdm_test.py" import openhtf as htf from openhtf.util import units @htf.measures( htf.Measurement("output_voltage_V") .in_range( minimum=4.9, maximum=5.1, marginal_minimum=4.92, marginal_maximum=5.08, ) .with_units(units.VOLT), htf.Measurement("current_draw_mA") .in_range(minimum=90, maximum=110) .with_units(units.MILLIAMPERE), ) def phase_production_test(test): """Production test with marginal bands for early warning.""" test.measurements.output_voltage_V = 5.01 test.measurements.current_draw_mA = 99.5 ``` Marginal limits are important for ZDM. A unit that passes within the marginal band technically passes, but it's trending toward a failure. TofuPilot flags marginal results separately, giving you an early warning before defects start escaping. ```python filename="zdm_test.py" from tofupilot.openhtf import TofuPilot test = htf.Test(phase_production_test) with TofuPilot(test): test.execute(test_start=lambda: input("Scan serial: ")) ``` TofuPilot supports zero-defect goals by providing: - **Measurement distributions** showing how close you are to limits - **Marginal result tracking** for early warning - **Failure Pareto** prioritizing which defects to eliminate next - **Control charts** detecting process drift before it produces defects - **Yield trends** measuring progress toward zero-defect targets - **Root cause data** correlating failures with process variables ## The Cost of Zero Defects | Investment | Return | |-----------|--------| | Better incoming inspection | Fewer defective components in assembly | | In-process monitoring (SPI, AOI) | Catch defects before value is added | | Structured test data collection | Enable prediction and correlation analysis | | Root cause investigation for every failure | Permanent fixes, not band-aids | | Marginal limit monitoring | Catch drift before it becomes failure | Zero-defect manufacturing is a direction, not a destination. Every defect prevented is cheaper than a defect detected, which is cheaper than a defect shipped. The test system's job is to generate the data that makes prevention possible. ### Migrate from NI TestStand to Python URL: https://www.tofupilot.com/guides/how-to-migrate-from-ni-teststand-to-python-with-tofupilot A practical guide to replacing NI TestStand with Python and OpenHTF for manufacturing test, with step-by-step migration patterns and TofuPilot integration. NI TestStand costs $4,310/seat/year, locks you into Windows, and requires specialized engineers to maintain. Python with OpenHTF gives you the same test sequencing capabilities with open-source tooling, cross-platform support, and version control. TofuPilot replaces TestStand's database logging and Process Model with structured analytics out of the box. ## Why Teams Migrate The decision usually comes down to three factors: | Factor | TestStand | Python + OpenHTF | |--------|-----------|--------------------| | License cost | $4,310/seat/year | Free | | Platform | Windows only | Windows, Linux, macOS | | Version control | Binary .seq files, hard to diff | Plain .py files, full Git support | | CI/CD | Custom integrations needed | Native Python tooling | | Hiring | Requires TestStand-trained engineers | Any Python developer | | Test editor | Proprietary GUI | Any code editor | | Instrument drivers | NI VISA + IVI | PyVISA (same instruments) | | Data management | Complex DB schema + custom queries | TofuPilot (built-in analytics) | The migration doesn't have to happen all at once. Most teams run both systems in parallel during the transition, converting one test procedure at a time. ## TestStand Concepts in Python Every TestStand concept has a direct Python equivalent. This mapping covers the core building blocks. ### Sequences and Steps In TestStand, you build a sequence of steps in the sequence editor. In OpenHTF, you define phases as Python functions and pass them to a Test object. **TestStand:** ``` MainSequence ├── Setup (Precondition group) ├── PowerOnSelfTest (step) ├── FunctionalTest (step) └── Cleanup (Postcondition group) ``` **Python with OpenHTF:** ```python filename="migration/sequence_mapping.py" import openhtf as htf from openhtf.util import units from tofupilot.openhtf import TofuPilot def setup(test): """Equivalent to TestStand Setup group.""" pass # Initialize fixture, instruments, etc. @htf.measures( htf.Measurement("post_voltage") .in_range(4.8, 5.2) .with_units(units.VOLT), ) def power_on_self_test(test): """Equivalent to a TestStand NumericLimitTest step.""" voltage = 5.01 # Replace with instrument read test.measurements.post_voltage = voltage @htf.measures( htf.Measurement("firmware_crc") .equals("0xA3F7B2C1"), ) def functional_test(test): """Equivalent to a TestStand StringValueTest step.""" crc = "0xA3F7B2C1" # Replace with DUT query test.measurements.firmware_crc = crc def cleanup(test): """Equivalent to TestStand Cleanup group.""" pass # Disable outputs, close connections def main(): test = htf.Test( setup, power_on_self_test, functional_test, cleanup, procedure_id="FCT-001", part_number="PCBA-100", ) with TofuPilot(test): test.execute(test_start=lambda: input("Scan serial number: ")) if __name__ == "__main__": main() ``` ### Step Types TestStand has built-in step types: NumericLimitTest, StringValueTest, PassFailTest. OpenHTF uses `Measurement` objects with validators. | TestStand Step Type | OpenHTF Equivalent | |--------------------|----------------------| | NumericLimitTest | `Measurement("name").in_range(low, high).with_units(unit)` | | StringValueTest | `Measurement("name").equals("expected")` | | PassFailTest | `Measurement("name").equals(True)` | | MultipleNumericLimitTest | Multiple `Measurement` objects on the same phase | | NI Switch / Action | Plain Python function (no measurement decorator) | ### Sharing Data Between Phases TestStand uses FileGlobals, StationGlobals, and Locals to pass data between steps. In OpenHTF, use plugs with instance attributes to share data across phases. The plug persists for the entire test execution. ```python filename="migration/shared_data.py" from openhtf.plugs import BasePlug import openhtf as htf class CalibrationPlug(BasePlug): """Stores calibration data shared between phases.""" def setUp(self): self.offset = 0.0 def tearDown(self): pass @htf.plug(cal=CalibrationPlug) def phase_one(test, cal): """Store calibration data for later phases.""" cal.offset = 0.023 @htf.plug(cal=CalibrationPlug) def phase_two(test, cal): """Use calibration data from a previous phase.""" raw_reading = 3.31 # From instrument corrected = raw_reading - cal.offset ``` ### Code Modules TestStand uses Code Modules (DLLs, .NET assemblies, LabVIEW VIs) to interface with instruments and DUTs. OpenHTF uses Plugs, which are Python classes with automatic lifecycle management. ```python filename="migration/plug_mapping.py" import pyvisa from openhtf.plugs import BasePlug class MultimeterPlug(BasePlug): """Equivalent to a TestStand Code Module wrapping an instrument driver.""" def setUp(self): """Called once before the test. Like TestStand's Setup entry point.""" rm = pyvisa.ResourceManager() self.instr = rm.open_resource("TCPIP::192.168.1.100::INSTR") self.instr.timeout = 5000 def measure_voltage(self, channel=1): """Query the multimeter for a DC voltage reading.""" self.instr.write(f":CONF:VOLT:DC AUTO,(@{channel})") self.instr.write(":INIT") return float(self.instr.query(":FETCH?")) def tearDown(self): """Called after the test. Like TestStand's Cleanup entry point.""" self.instr.close() ``` ## Replace TestStand Database Logging TestStand's built-in database logger writes to SQL Server, Oracle, or Access using a fixed schema. The core tables (`UUT_RESULT`, `STEP_RESULT`, `PROP_RESULT`, `PROP_NUMERICLIMIT`, `PROP_NUMERIC`) require 5-6 JOINs for a simple measurement query. ### The TestStand Database Schema Problem A typical query to get one unit's test results in TestStand's database: ```sql filename="teststand_query.sql" -- 6-table JOIN to get measurements for one serial number SELECT u.UUT_SERIAL_NUMBER, u.UUT_STATUS, s.STEP_NAME, s.STATUS, n.DATA AS measured_value, nl.LOW AS lower_limit, nl.HIGH AS upper_limit, nl.UNITS FROM UUT_RESULT u JOIN STEP_RESULT s ON s.UUT_RESULT = u.ID JOIN PROP_RESULT p ON p.STEP_RESULT = s.ID JOIN PROP_NUMERICLIMIT nl ON nl.PROP_RESULT = p.ID JOIN PROP_NUMERIC n ON n.PROP_RESULT = p.ID WHERE u.UUT_SERIAL_NUMBER = 'SN-5001' ORDER BY u.START_DATE_TIME DESC ``` This schema is rigid. Adding custom metadata (firmware version, station ID, operator) means modifying the Process Model's database mapping, which is fragile across TestStand upgrades. TestStand doesn't include analytics. FPY trends, Cpk, control charts, and failure Pareto all require custom SQL or a third-party tool. ### TofuPilot Replaces All of This With OpenHTF + TofuPilot, you don't manage a database. Measurements flow directly from your test code: ```python filename="migration/tofupilot_replacement.py" import openhtf as htf from openhtf.util import units from tofupilot.openhtf import TofuPilot @htf.measures( htf.Measurement("rail_3v3") .in_range(3.2, 3.4) .with_units(units.VOLT), htf.Measurement("current_draw") .in_range(0.1, 0.5) .with_units(units.AMPERE), ) def power_test(test): test.measurements.rail_3v3 = 3.31 test.measurements.current_draw = 0.25 def main(): test = htf.Test(power_test, procedure_id="FCT-001", part_number="PCBA-100") with TofuPilot(test): test.execute(test_start=lambda: input("Scan serial: ")) if __name__ == "__main__": main() ``` No database connection code. No SQL. No schema maintenance. | TestStand Database | TofuPilot | |-------------------|-----------| | 6-table JOIN for one unit's results | Search by serial number, get full history | | Custom SQL for FPY | FPY trends updated in real time | | No Cpk without custom code | Cpk per measurement, automatic | | No control charts | Control charts with UCL/LCL | | No failure Pareto | Failure Pareto with drill-down | | Schema locked to NI's design | Structured data, REST API access | | SQL Server/Oracle/Access | Cloud or self-hosted | | One site at a time | Multi-site from day one | TofuPilot tracks FPY, Cpk, throughput, and failure analysis automatically. Open the Analytics tab to see trends for any procedure. ### Import Historical TestStand Data You have years of test data in TestStand's database. TofuPilot's Python SDK lets you import it: ```python filename="migration/import_teststand_history.py" import pyodbc from tofupilot import TofuPilotClient client = TofuPilotClient() conn = pyodbc.connect( "DRIVER={SQL Server};" "SERVER=teststand-db;" "DATABASE=TestStandResults;" ) cur = conn.cursor() cur.execute(""" SELECT u.ID, u.UUT_SERIAL_NUMBER, u.UUT_STATUS, u.START_DATE_TIME, u.EXECUTION_TIME, s.STEP_NAME, s.STATUS, n.DATA, nl.LOW, nl.HIGH, nl.UNITS FROM UUT_RESULT u JOIN STEP_RESULT s ON s.UUT_RESULT = u.ID LEFT JOIN PROP_RESULT p ON p.STEP_RESULT = s.ID LEFT JOIN PROP_NUMERICLIMIT nl ON nl.PROP_RESULT = p.ID LEFT JOIN PROP_NUMERIC n ON n.PROP_RESULT = p.ID ORDER BY u.START_DATE_TIME """) current_uut_id = None steps = [] for row in cur: uut_id, serial, status, started, duration, step_name, step_status, value, low, high, unit = row if current_uut_id and current_uut_id != uut_id: client.create_run( procedure_id="FCT-001", unit_under_test={"serial_number": prev_serial}, run_passed=(prev_status == "Passed"), started_at=prev_started.isoformat(), duration=prev_duration, steps=steps, ) steps = [] current_uut_id = uut_id prev_serial = serial prev_status = status prev_started = started prev_duration = duration step = next((s for s in steps if s["name"] == step_name), None) if not step: step = {"name": step_name, "step_passed": step_status == "Passed", "measurements": []} steps.append(step) if value is not None: step["measurements"].append({ "name": step_name, "measured_value": value, "unit": unit or "", "lower_limit": low, "upper_limit": high, }) conn.close() ``` Adapt the connection string and procedure_id to match your setup. Run this once to backfill your TofuPilot workspace with historical trends. ## Process Models TestStand's Process Model handles serial number input, report generation, and database logging. With TofuPilot, you get all three: - **Serial number input** via `test.execute(test_start=lambda: input("Scan: "))` or custom UI - **Report generation** is automatic (every run gets a detailed page in TofuPilot) - **Database logging** happens on every run (measurements, limits, pass/fail, attachments) - **Analytics** (FPY, Cpk, control charts) are computed automatically from your data No custom output callbacks or database connectors needed. ## Migration Strategy ### Phase 1: Parallel Operation (Week 1-2) Keep TestStand running. Set up a Python environment alongside it. Convert one simple test procedure (the easiest one, fewest instruments). Run both versions on the same DUTs to validate results match. ``` Test Station ├── TestStand (existing tests) └── Python + OpenHTF (new test, same DUT) └── TofuPilot (data logging) ``` ### Phase 2: Instrument Drivers (Week 2-4) Convert TestStand Code Modules to Python plugs. Most NI instruments work with PyVISA (same VISA layer TestStand uses). Third-party instruments with SCPI support work out of the box. | TestStand Driver | Python Equivalent | |-----------------|-------------------| | NI VISA / IVI | PyVISA + pyvisa-py or NI-VISA backend | | NI DAQmx | nidaqmx (official NI Python package) | | NI Switch | niswitch (official NI Python package) | | NI DMM | nidmm (official NI Python package) | | Serial / UART | pyserial | | Custom DLL | ctypes or cffi | ### Phase 3: Full Conversion (Week 4-8) Convert remaining test procedures one at a time. Start with the highest-volume procedures (biggest impact on throughput). Keep TestStand as fallback until each procedure is validated. ### Phase 4: Decommission (Week 8+) Once all procedures are running in Python and validated against TestStand results, decommission TestStand. Cancel the license renewals. Archive the .seq files. ## What You Gain After migration, your test infrastructure looks different: - **Test scripts in Git.** Full diff history, code review on test changes, branching for new product variants. - **CI/CD for tests.** Run linting, type checking, and unit tests on test scripts before deploying to stations. - **Any OS.** Test stations can run Linux (cheaper, more stable for long-running production). - **Any editor.** VS Code, PyCharm, vim. No proprietary IDE. - **Analytics from day one.** TofuPilot gives you FPY, Cpk, control charts, and failure Pareto without building custom database integrations. - **Half the hiring pool opens up.** Any Python developer can contribute to test development. ## Common Pitfalls ### Don't convert everything at once The biggest migration risk is trying to convert all procedures simultaneously. Convert one, validate it, move to the next. Parallel operation is your safety net. ### Don't skip instrument validation PyVISA talks to the same instruments, but timing and trigger behavior can differ from NI VISA drivers. Validate measurements match between the old and new systems on the same DUT. A 0.1% measurement difference matters when your limits are tight. ### Don't lose your test data history Export historical data from TestStand's database before decommissioning. Use the import script above to backfill TofuPilot so you maintain traceability and trend analysis across the migration boundary. ### What Is Gage R&R and MSA URL: https://www.tofupilot.com/guides/what-is-grr-and-msa-with-tofupilot Gage R&R and MSA quantify measurement system variation. Learn how to assess your test equipment and track measurement reliability with TofuPilot. # What Is GRR and MSA with TofuPilot Gage Repeatability and Reproducibility (GRR) measures how much of your test variation comes from the measurement system itself. Measurement System Analysis (MSA) is the broader evaluation that includes GRR plus bias, linearity, and stability. This guide covers how GRR and MSA work, how to run a GRR study, and how to use TofuPilot data to monitor measurement system health. ## Why Measurement Variation Matters Every test measurement has three sources of variation: | Source | What It Is | |--------|-----------| | Part-to-part | Real differences between units (this is what you want to measure) | | Repeatability | Variation when the same operator measures the same part multiple times | | Reproducibility | Variation when different operators or stations measure the same part | If your measurement system contributes too much variation, you can't trust your pass/fail decisions. Good units get rejected (false failures). Bad units pass (escapes). ## GRR Acceptance Criteria | GRR % of Tolerance | Assessment | |--------------------|-----------| | Below 10% | Measurement system is acceptable | | 10% to 30% | May be acceptable depending on application | | Above 30% | Measurement system needs improvement | GRR is expressed as a percentage of the specification tolerance. A GRR of 20% means the measurement system uses up 20% of your tolerance band with its own noise. ## How a GRR Study Works A crossed GRR study uses multiple operators, multiple parts, and multiple trials: | Parameter | Typical Value | |-----------|--------------| | Operators (or stations) | 2-3 | | Parts | 10 (spanning the range of production variation) | | Trials per part per operator | 2-3 | | Total measurements | 40-90 | Each operator measures each part multiple times, in random order. The data is analyzed using ANOVA or the range method to separate repeatability and reproducibility. ## Prerequisites - Python 3.10+ - OpenHTF installed (`pip install openhtf`) - TofuPilot Python SDK installed (`pip install tofupilot`) ## Step 1: Collect GRR Data Run the same measurement on the same set of parts multiple times. Use TofuPilot to log every measurement with the part identifier and trial number. ```python filename="grr_study.py" import openhtf as htf from openhtf.util import units @htf.measures( htf.Measurement("voltage_reading_V") .with_units(units.VOLT), htf.Measurement("operator").with_units(units.UNITLESS), htf.Measurement("trial").with_units(units.UNITLESS), ) def phase_grr_measurement(test): """Measure voltage for GRR study. Record operator and trial.""" test.measurements.voltage_reading_V = 5.003 test.measurements.operator = 1 test.measurements.trial = 1 ``` ## Step 2: Log to TofuPilot Each measurement becomes a test run. The serial number identifies the part. Run each part-operator-trial combination as a separate test execution. ```python filename="grr_study.py" from tofupilot.openhtf import TofuPilot test = htf.Test(phase_grr_measurement) with TofuPilot(test): test.execute(test_start=lambda: input("Scan GRR part serial: ")) ``` ## Step 3: Analyze the Results After collecting all measurements, export the data from TofuPilot and compute the GRR metrics. The key calculations: | Metric | Formula | |--------|---------| | Repeatability (EV) | Average range within operator x d2 constant | | Reproducibility (AV) | Range of operator averages x d2 constant, corrected for sample size | | GRR | Square root of (EV squared + AV squared) | | GRR % | (GRR / tolerance) x 100 | | Number of distinct categories (ndc) | (Part variation / GRR) x 1.41 | An ndc below 5 means the measurement system can't distinguish enough categories of parts. You need at least 5 for a capable system. ## Beyond GRR: Full MSA MSA includes four additional evaluations beyond GRR: | Study | What It Measures | When to Run | |-------|-----------------|-------------| | Bias | Difference between measured average and known reference | At calibration | | Linearity | How bias changes across the measurement range | At calibration | | Stability | How measurements drift over time | Ongoing monitoring | | Resolution | Smallest increment the system can detect | At setup | ## Using TofuPilot for Ongoing MSA You don't need to run formal GRR studies constantly. TofuPilot's measurement distribution data gives you ongoing visibility: - **Measurement distributions** for the same part number show total variation over time - **Station comparison** shows whether different stations give different results (reproducibility signal) - **Control charts** detect when measurement drift begins (stability monitoring) - **Marginal results** flag when measurements are close to limits, which could indicate measurement system issues rather than part issues When you see unexpected variation in TofuPilot, run a formal GRR study to quantify the source. If the measurement system is the problem, recalibrate or upgrade the instrument before tightening limits. ### Hardware Test Infrastructure with TofuPilot URL: https://www.tofupilot.com/guides/hardware-test-infrastructure-with-tofupilot Learn how to build scalable hardware test infrastructure using TofuPilot as the data backbone for test stations, instruments, and automation. # Hardware Test Infrastructure with TofuPilot Test infrastructure is everything between the engineer's test script and the data that drives quality decisions. Instruments, fixtures, stations, software frameworks, data pipelines. TofuPilot sits at the center as the data backbone, connecting your test stations to dashboards, analytics, and traceability. ## What Hardware Test Infrastructure Looks Like A production test setup has multiple layers: ``` ┌────────────────────────────────────────────┐ │ TofuPilot (Cloud) │ │ Dashboards · Analytics · Traceability │ ├────────────────────────────────────────────┤ │ Test Framework Layer │ │ OpenHTF · pytest · Custom Python │ ├────────────────────────────────────────────┤ │ Instrument Layer │ │ DMMs · Oscilloscopes · Power Supplies │ │ DAQs · Spectrum Analyzers · Load Banks │ ├────────────────────────────────────────────┤ │ Fixture Layer │ │ Pogo pin fixtures · Cable harnesses │ │ Pneumatic actuators · Thermal chambers │ ├────────────────────────────────────────────┤ │ DUT (Device Under Test) │ └────────────────────────────────────────────┘ ``` Each layer has its own concerns. TofuPilot handles the top layer: collecting, storing, and analyzing the data that flows up from the test stack. ## Station Architecture A well-designed test station separates concerns: | Component | Responsibility | Tools | |-----------|---------------|-------| | Test logic | What to test and in what order | OpenHTF, pytest, custom Python | | Instrument control | Communicating with test equipment | PyVISA, python-ivi, vendor drivers | | Fixture control | Engaging/disengaging the DUT | GPIO, serial, pneumatic controllers | | Data capture | Recording measurements | TofuPilot client | | Operator interface | Scan serial, display results | OpenHTF UI, custom GUI | ### Minimal Station Setup ```python filename="minimal_station.py" from tofupilot import TofuPilotClient import pyvisa # Instrument layer rm = pyvisa.ResourceManager() dmm = rm.open_resource("TCPIP::192.168.1.10::INSTR") psu = rm.open_resource("TCPIP::192.168.1.11::INSTR") # Test logic def run_functional_test(serial_number): psu.write("OUTP ON") vcc = float(dmm.query("MEAS:VOLT:DC?")) current = float(dmm.query("MEAS:CURR:DC?")) psu.write("OUTP OFF") passed = 3.25 <= vcc <= 3.35 and 30 <= current <= 60 # Data capture client = TofuPilotClient() client.create_run( procedure_id="BOARD-FUNCTIONAL", unit_under_test={"serial_number": serial_number}, run_passed=passed, steps=[{ "name": "Power Rails", "step_type": "measurement", "status": 3.25 <= vcc <= 3.35, "measurements": [ {"name": "vcc_3v3", "value": vcc, "unit": "V", "limit_low": 3.25, "limit_high": 3.35}, {"name": "idle_current_ma", "value": current * 1000, "unit": "mA", "limit_low": 30, "limit_high": 60}, ], }], ) return passed # Operator interface while True: serial = input("Scan serial (or 'q' to quit): ") if serial.lower() == "q": break result = run_functional_test(serial) print(f"{'PASS' if result else 'FAIL'}") ``` ## Scaling from One Station to Many ### Identical Stations When you add stations for the same test, keep the test code identical across all stations. Use the same procedure ID. TofuPilot distinguishes stations automatically. ```python filename="station_config.py" import socket # Station identification STATION_ID = socket.gethostname() # Each station PC has a unique hostname # Same test code, same procedure, different station client.create_run( procedure_id="BOARD-FUNCTIONAL", unit_under_test={"serial_number": serial}, run_passed=passed, steps=steps, ) # TofuPilot tracks which station ran each test ``` ### Different Test Stages Production flows typically have multiple test stages. Each stage gets its own procedure. ``` Assembly Line: Station 1-4: ICT (In-Circuit Test) → procedure: "ICT-V2" Station 5-6: Functional Test → procedure: "FUNC-TEST-V3" Station 7: Burn-In → procedure: "BURN-IN-24H" Station 8-10: Final Test → procedure: "FINAL-TEST-V2" ``` TofuPilot links all test stages for a given serial number, creating a complete manufacturing test history. ## Instrument Management ### Common Instrument Types | Instrument | What it measures | Interface | |-----------|-----------------|-----------| | DMM (Digital Multimeter) | Voltage, current, resistance | GPIB, LAN, USB | | Oscilloscope | Waveforms, timing | LAN, USB | | Power Supply | (Provides power, measures output) | GPIB, LAN, USB | | DAQ (Data Acquisition) | Multi-channel analog/digital | USB, PCIe | | Spectrum Analyzer | Frequency content | GPIB, LAN | | Load Bank | (Simulates load conditions) | Serial, LAN | | Environmental Chamber | Temperature, humidity | Serial, LAN | ### Instrument Connection Patterns ```python filename="instrument_pool.py" import pyvisa class InstrumentPool: """Manage instrument connections for a test station.""" def __init__(self): self.rm = pyvisa.ResourceManager() self._instruments = {} def get(self, name, address): if name not in self._instruments: self._instruments[name] = self.rm.open_resource(address) return self._instruments[name] def close_all(self): for inst in self._instruments.values(): inst.close() # Usage pool = InstrumentPool() dmm = pool.get("dmm", "TCPIP::192.168.1.10::INSTR") psu = pool.get("psu", "TCPIP::192.168.1.11::INSTR") ``` ## Infrastructure Monitoring Your test infrastructure is itself something that needs monitoring. TofuPilot's dashboards reveal infrastructure health: | Metric | What it indicates | |--------|------------------| | FPY per station | Station hardware health | | Cycle time per station | Instrument/fixture performance | | Measurement variance per station | Fixture contact quality | | Uptime per station | Reliability of test PC/software | When one station's FPY drops while others stay stable, the problem is the station, not the product. Investigate the fixture, instruments, and cabling. ## Infrastructure Best Practices | Practice | Why | |----------|-----| | Version your test code | Know exactly what test ran on each unit | | Use the same procedure ID across identical stations | Enables cross-station comparison | | Store instrument calibration dates | Know when calibration might affect results | | Keep fixtures as simple as possible | Fewer moving parts = fewer failure modes | | Monitor cycle time | It's a free health indicator | | Separate test logic from instrument drivers | Swap instruments without rewriting tests | | Use IP-based instrument connections | More reliable than USB for production | ## What TofuPilot Handles vs. What You Handle | TofuPilot handles | You handle | |------------------|-----------| | Data storage and indexing | Test code and logic | | Dashboards and analytics | Instrument drivers | | Traceability and search | Fixture design and maintenance | | Yield calculations | Station hardware setup | | Measurement trending | Network connectivity | | Cross-station comparison | Operator training | TofuPilot is the data layer. You bring the test layer. Together they form a complete test infrastructure. ### IPC-A-610 Inspection Tracking with TofuPilot URL: https://www.tofupilot.com/guides/ipc-a-610-inspection-tracking-with-tofupilot Learn how to track IPC-A-610 visual inspection results for PCBA acceptability using TofuPilot's structured test records. # IPC-A-610 Inspection Tracking with TofuPilot IPC-A-610 is the acceptability standard for electronic assemblies. It defines what "good" looks like for solder joints, component placement, cleanliness, and mechanical assembly. TofuPilot tracks inspection results systematically instead of relying on paper checklists. ## What IPC-A-610 Covers IPC-A-610 classifies defects into three product classes: | Class | Application | Acceptance criteria | |-------|-------------|-------------------| | Class 1 | General electronics (consumer) | Least strict | | Class 2 | Dedicated service electronics (industrial) | Moderate | | Class 3 | High-performance electronics (medical, aerospace, military) | Most strict | Inspection criteria include: | Category | Examples | |----------|---------| | Solder joints | Wetting, fillets, bridges, cold joints | | Component placement | Alignment, orientation, polarity | | Cleanliness | Flux residue, contamination, corrosion | | Mechanical | Wire routing, strain relief, conformal coating | | Marking | Labels, part numbers, date codes | ## Logging Inspection Results to TofuPilot ### Per-Board Inspection ```python filename="ipc610_inspection.py" from tofupilot import TofuPilotClient client = TofuPilotClient() def log_inspection(serial, inspector, defects): """Log IPC-A-610 inspection results.""" measurements = [ {"name": "inspector_id", "value": inspector, "unit": ""}, {"name": "ipc_class", "value": 2, "unit": "class"}, {"name": "total_defects", "value": len(defects), "unit": "count", "limit_high": 0}, ] # Log each defect category defect_categories = { "solder": 0, "placement": 0, "cleanliness": 0, "mechanical": 0, "marking": 0, } for d in defects: if d["category"] in defect_categories: defect_categories[d["category"]] += 1 for cat, count in defect_categories.items(): measurements.append({ "name": f"defects_{cat}", "value": count, "unit": "count", "limit_high": 0, }) passed = len(defects) == 0 client.create_run( procedure_id="IPC610-VISUAL-INSPECTION", unit_under_test={ "serial_number": serial, "part_number": "MAIN-BOARD-V4", }, run_passed=passed, steps=[{ "name": "IPC-A-610 Class 2 Inspection", "step_type": "measurement", "status": passed, "measurements": measurements, }], ) # Example: board with two defects log_inspection( serial="PCB-2025-04521", inspector="OP-012", defects=[ {"category": "solder", "location": "U12-pin3", "type": "insufficient_wetting"}, {"category": "cleanliness", "location": "J5-area", "type": "flux_residue"}, ], ) ``` ### Defect Classification Log specific defect types to build a defect pareto. ```python filename="defect_logging.py" # Detailed defect logging defect_types = { "solder_bridge": "Solder bridge between adjacent pins", "cold_joint": "Cold or disturbed solder joint", "insufficient_wetting": "Insufficient solder wetting on pad or lead", "solder_ball": "Loose solder ball on board surface", "tombstone": "Component standing on end (tombstoning)", "misalignment": "Component offset from pad center", "wrong_polarity": "Polarized component installed backwards", "missing_component": "Component not placed", "flux_residue": "Excessive flux residue after cleaning", "damaged_component": "Component body cracked or chipped", } # Log each defect with its type for defect in board_defects: measurements.append({ "name": f"defect_{defect['type']}", "value": 1, "unit": "count", "limit_high": 0, }) ``` ## Defect Pareto Analysis TofuPilot's analytics show you the most common defect types across production. | Rank | Defect type | Count | Percentage | |------|------------|-------|------------| | 1 | Insufficient wetting | 45 | 32% | | 2 | Solder bridge | 28 | 20% | | 3 | Flux residue | 22 | 16% | | 4 | Misalignment | 15 | 11% | | 5 | Cold joint | 12 | 9% | | - | All others | 18 | 13% | Fix the top defect type first. Insufficient wetting (32%) is likely a solder paste issue (stencil wear, paste viscosity, or reflow profile). ## Inspector Consistency Track defect detection rates by inspector to identify training needs. ```python filename="inspector_analysis.py" from tofupilot import TofuPilotClient client = TofuPilotClient() runs = client.get_runs( procedure_id="IPC610-VISUAL-INSPECTION", limit=1000, ) # Group by inspector inspector_stats = {} for run in runs: for step in run.get("steps", []): for m in step.get("measurements", []): if m["name"] == "inspector_id": inspector = m["value"] if inspector not in inspector_stats: inspector_stats[inspector] = {"total": 0, "rejected": 0} inspector_stats[inspector]["total"] += 1 if not run["run_passed"]: inspector_stats[inspector]["rejected"] += 1 for inspector, stats in inspector_stats.items(): reject_rate = stats["rejected"] / stats["total"] * 100 print(f"Inspector {inspector}: {stats['total']} boards, {reject_rate:.1f}% rejection rate") ``` If Inspector A rejects 8% and Inspector B rejects 2% on the same product, either A is too strict or B is missing defects. Calibrate inspectors using reference boards with known defects. ## Connecting Inspection to Electrical Test The real power is correlating visual inspection results with electrical test results. If boards that pass visual inspection but fail functional test at a specific measurement, there may be a defect type that visual inspection isn't catching. Conversely, if visually rejected boards are reworked and always pass functional test, the visual criteria may be too strict for your product class. TofuPilot links both inspection and electrical test results to the same serial number, making this correlation straightforward. ## AOI Integration Automated Optical Inspection (AOI) systems can push results to TofuPilot the same way manual inspections do. ```python filename="aoi_upload.py" # Parse AOI machine output and upload to TofuPilot import json with open("aoi_results.json") as f: aoi_data = json.load(f) for board in aoi_data["boards"]: defects = board.get("defects", []) client.create_run( procedure_id="AOI-INSPECTION", unit_under_test={"serial_number": board["serial"]}, run_passed=len(defects) == 0, steps=[{ "name": "AOI Scan", "step_type": "measurement", "status": len(defects) == 0, "measurements": [ {"name": "defect_count", "value": len(defects), "unit": "count", "limit_high": 0}, {"name": "scan_coverage_pct", "value": board.get("coverage", 100), "unit": "%"}, ], }], ) ``` Combine AOI data with manual inspection data and electrical test data in TofuPilot for a complete quality picture of every board. ### Test Acceleration for Hardware Teams URL: https://www.tofupilot.com/guides/test-acceleration-for-hardware-teams-with-tofupilot Learn how to accelerate hardware test cadence using TofuPilot to identify bottlenecks, reduce debug time, and optimize test coverage. # Test Acceleration for Hardware Teams with TofuPilot Hardware teams don't need to test more. They need to test faster. The bottleneck isn't usually the test itself. It's the time spent searching for data, debugging failures manually, and waiting for reports. TofuPilot eliminates these delays. ## Where Hardware Teams Lose Time | Activity | Typical time | With TofuPilot | |----------|-------------|----------------| | Finding test data for a specific unit | 15-30 min | 10 seconds | | Comparing passing vs. failing runs | 1-2 hours | 2 minutes | | Building a weekly quality report | 3-4 hours | Already done (live dashboard) | | Correlating failures with component lots | 1-2 days | 15 minutes | | Answering "What's our yield?" | 30 min (pull data, calculate) | Glance at dashboard | | Debugging a field return | 2-4 hours | 5 minutes (search by serial) | The test run itself might take 60 seconds. Everything around it takes hours or days. That's where acceleration happens. ## Accelerating Failure Debug ### Before TofuPilot 1. Unit fails on the test station 2. Operator calls the test engineer 3. Test engineer walks to the station, looks at the screen 4. Test engineer manually records the failing measurements 5. Test engineer goes back to their desk, opens old data files to compare 6. Test engineer emails the design team with findings 7. Back and forth continues ### With TofuPilot 1. Unit fails on the test station 2. Test engineer opens TofuPilot, filters to the failed run 3. Compares measurements against recent passing runs in one click 4. Identifies the anomalous measurement 5. Checks the measurement trend to see when it started 6. Shares the dashboard link with the design team Step 2-6 takes 5 minutes. The old way takes half a day. ## Accelerating Test Development ### Data-Driven Limit Setting Instead of guessing limits from datasheets, run 50 units and use the actual distribution to set limits. ```python filename="set_limits_from_data.py" import numpy as np from tofupilot import TofuPilotClient client = TofuPilotClient() # Get pilot production data runs = client.get_runs( procedure_id="PILOT-FUNCTIONAL", limit=50, ) # Extract values for each measurement measurements = {} for run in runs: for step in run.get("steps", []): for m in step.get("measurements", []): if m["name"] not in measurements: measurements[m["name"]] = [] measurements[m["name"]].append(m["value"]) # Calculate recommended limits (mean +/- 4 sigma) for name, values in measurements.items(): arr = np.array(values) mean = np.mean(arr) std = np.std(arr, ddof=1) print(f"{name}: mean={mean:.4f}, std={std:.4f}") print(f" Recommended limits: [{mean - 4*std:.4f}, {mean + 4*std:.4f}]") ``` This replaces weeks of limit tuning with 30 minutes of data analysis. ### Identifying Redundant Tests Not every measurement adds value. Some measurements never fail. Some always correlate perfectly with another measurement (testing the same thing twice). Pull your measurement data from TofuPilot and check: | Check | Action | |-------|--------| | Measurement never fails (100% pass rate over 1000+ units) | Consider removing or widening limits | | Two measurements always fail together (r > 0.95) | One might be redundant | | Measurement adds 10s to cycle time but catches 0.01% of defects | Consider removing from production test | Cutting redundant measurements directly reduces cycle time and increases throughput. ## Accelerating Yield Improvement ### The Improvement Loop ``` Measure → Analyze → Improve → Verify ↑ │ └──────────────────────────────┘ ``` TofuPilot accelerates every step: 1. **Measure**: Automatic data collection from every station 2. **Analyze**: Dashboards show yield trends, failure paretos, and measurement distributions 3. **Improve**: Data points to the root cause, so fixes are targeted 4. **Verify**: Before/after comparison confirms the fix worked Without centralized data, steps 1 and 2 take days. With TofuPilot, they're instant. ### Prioritizing Improvements The failure pareto in TofuPilot shows you where to focus. Fix the #1 failure mode first. It has the biggest yield impact. Don't try to fix everything at once. Fix one thing, verify the improvement in TofuPilot, then move to the next. ## Accelerating NPI (New Product Introduction) During NPI, test development and product development happen in parallel. Every design revision needs updated tests. Every test result informs the next design revision. TofuPilot accelerates this loop by: - Storing results from every prototype build - Comparing measurements across design revisions - Showing which design changes improved (or regressed) specific measurements - Providing immediate visibility into whether a new build meets specs ### Comparing Design Revisions ```python filename="revision_comparison.py" from tofupilot import TofuPilotClient client = TofuPilotClient() # Get runs from two design revisions rev_a_runs = client.get_runs(procedure_id="PROTO-FUNCTIONAL", limit=20) rev_b_runs = client.get_runs(procedure_id="PROTO-FUNCTIONAL", limit=20) # Compare measurement means # Filter by part_number or date range to separate revisions ``` If Rev B has 15% lower idle current and 5% tighter voltage regulation, the design change worked. If thermal measurements regressed, the new layout needs attention. TofuPilot shows this in minutes, not days. ## The Compound Effect Each individual acceleration seems small: 5 minutes saved here, 30 minutes there. But across a team of 5 test engineers running 10 debug sessions per week, the compound effect is significant. | Savings per debug session | 2 hours | |--------------------------|---------| | Debug sessions per week | 50 (5 engineers x 10) | | Weekly time saved | 100 hours | | Monthly time saved | 400 hours | That's 400 engineering hours per month redirected from data gathering to actual engineering work: improving designs, optimizing processes, and shipping better products. ### Hardware Telemetry Analysis with TofuPilot URL: https://www.tofupilot.com/guides/hardware-telemetry-analysis-with-tofupilot Learn how to collect, store, and analyze hardware sensor telemetry data using TofuPilot's measurement arrays and dashboards. # Hardware Telemetry Analysis with TofuPilot Sensor data from hardware tests is only useful if you can find patterns across thousands of runs. TofuPilot ingests multi-dimensional telemetry, indexes it automatically, and gives you dashboards that surface trends before they become production issues. ## What Hardware Telemetry Looks Like in Practice A single hardware test run can produce hundreds of sensor readings: temperature curves, voltage traces, vibration spectra, pressure waveforms. Without a structured system, this data ends up in CSV files on shared drives, impossible to query at scale. TofuPilot treats every sensor measurement as a first-class object. Each reading gets a name, unit, limits, and an optional array dimension for time-series or multi-axis data. ## Ingesting Sensor Data ### OpenHTF Users OpenHTF measurements flow into TofuPilot automatically. Multi-dimensional arrays (1D waveforms, 2D matrices, ND tensors) are processed without code changes. ```python filename="telemetry_test.py" import openhtf as htf from tofupilot.openhtf import TofuPilotClient @htf.measures( htf.Measurement("temperature_curve").with_dimensions("time_s"), htf.Measurement("vibration_spectrum").with_dimensions("freq_hz"), ) def sensor_sweep(test): for t in range(100): test.measurements.temperature_curve[t] = read_thermocouple() for f in range(500): test.measurements.vibration_spectrum[f] = read_accelerometer_fft(f) def main(): test = htf.Test(sensor_sweep) test.add_output_callbacks(TofuPilotClient()) test.execute(lambda: "DUT-001") ``` ### Python Client Users Pass lists or NumPy arrays directly in your measurement payloads. ```python filename="upload_telemetry.py" from tofupilot import TofuPilotClient import numpy as np client = TofuPilotClient() voltage_trace = np.sin(np.linspace(0, 2 * np.pi, 1000)).tolist() client.create_run( procedure_id="PSU-RIPPLE-TEST", unit_under_test={"serial_number": "PSU-2025-0042"}, steps=[{ "name": "Output Ripple", "step_type": "measurement", "status": True, "measurements": [{ "name": "ripple_voltage_mv", "value": voltage_trace, "unit": "mV", "limit_low": -50, "limit_high": 50, }], }], ) ``` ## Querying Telemetry at Scale Once ingested, every measurement is queryable through the TofuPilot dashboard. Filter by procedure, unit serial number, date range, or pass/fail status. | Query | What it shows | |-------|--------------| | All runs for `PSU-RIPPLE-TEST` last 7 days | Ripple voltage trends across production | | Failed runs with `temperature_curve` out of limits | Units that exceeded thermal specs | | Measurement distribution for `vibration_spectrum` | Histogram of vibration amplitudes across fleet | ## Setting Up Alerts on Telemetry Drift TofuPilot tracks measurement distributions over time. When a sensor reading starts drifting toward its limits, you can catch it before it causes failures. 1. Open the procedure dashboard for your test 2. Select the measurement you want to monitor 3. Set warning thresholds at 80% of your spec limits 4. TofuPilot flags runs where readings cross the warning zone This is especially useful for temperature sensors, where gradual calibration drift can go unnoticed until units start failing in the field. ## Real-World Example: Thermal Chamber Telemetry A medical device manufacturer runs thermal cycling tests across 8 temperature zones. Each test produces 8 time-series measurements (one per zone) with 500 data points each. Before TofuPilot, engineers downloaded CSV files from each chamber controller, merged them in Excel, and manually checked limits. With TofuPilot, all 8 waveforms upload automatically at the end of each cycle, limits are checked server-side, and the dashboard shows zone-by-zone trends across the full production history. The result: limit violations that previously took hours to find now surface in seconds. ### What Is Shift-Left Quality in Manufacturing URL: https://www.tofupilot.com/guides/what-is-shift-left-quality-in-manufacturing Shift-left quality moves defect detection earlier in the manufacturing process. Learn how it reduces cost and how test data enables it. # What Is Shift-Left Quality in Manufacturing Shift-left quality means moving defect detection earlier in the manufacturing process. The further left (earlier) you catch a problem, the cheaper it is to fix. A component defect caught at incoming inspection costs pennies to address. The same defect caught in the field costs hundreds of dollars. This guide covers how shift-left quality works, what it costs at each stage, and how test data enables the shift. ## The Cost of Finding Defects Late | Stage | Relative Cost to Fix | Example | |-------|---------------------|---------| | Design (simulation) | 1x | Catch a voltage margin issue in SPICE | | Incoming inspection (IQC) | 10x | Reject a bad component lot before assembly | | In-process (IPQC/AOI) | 25x | Rework a solder bridge before final assembly | | End-of-line test (EOL) | 50x | Scrap or rework a fully assembled unit | | Field return | 500-1000x | Warranty repair, shipping, customer impact | The 10x rule is well-established in manufacturing: every stage you delay detection, the cost increases by roughly an order of magnitude. ## What Shifting Left Looks Like | Before (Right-Heavy) | After (Shift-Left) | |----------------------|-------------------| | Test everything at EOL | Add measurements at IQC and IPQC | | Find solder defects at functional test | Catch them at AOI/SPI after reflow | | Discover component issues during assembly | Screen components at incoming inspection | | Learn about failure modes from field returns | Detect them during DVT and ORT | | Set limits from engineering judgment | Set limits from upstream process data | Shifting left doesn't mean removing end-of-line testing. It means adding detection points earlier so fewer defects reach the final test stage. ## The Data Requirement Shift-left quality requires data from every stage. You can't correlate upstream measurements with downstream failures if you're only collecting data at EOL. | Stage | Data Needed | |-------|------------| | IQC | Component measurements, supplier lot info, CoC data | | IPQC | SPI paste volume, AOI defect counts, process parameters | | FCT | Electrical measurements with limits and units | | EOL | Full functional test results per serial number | | Field | Return reason codes, failure analysis results | When all stages feed data into one platform, correlations become visible. You can answer questions like: "Do units from supplier A's lot 2024-47 have higher EOL failure rates than supplier B's lot 2024-48?" ## Prerequisites - Python 3.10+ - OpenHTF installed (`pip install openhtf`) - TofuPilot Python SDK installed (`pip install tofupilot`) ## Step 1: Add Upstream Test Points Don't wait until end-of-line to take measurements. Add test phases at earlier stages. ```python filename="iqc_test.py" import openhtf as htf from openhtf.util import units @htf.measures( htf.Measurement("capacitor_esr_mOhm") .in_range(maximum=45) .with_units(units.OHM), htf.Measurement("resistance_ohm") .within_percent(4700, 5) .with_units(units.OHM), ) def phase_iqc_check(test): """Incoming component check. Catches bad parts before assembly.""" test.measurements.capacitor_esr_mOhm = 32.1 test.measurements.resistance_ohm = 4720 ``` ## Step 2: Log Every Stage to TofuPilot Each stage uploads results independently. TofuPilot links them by serial number or lot code. ```python filename="iqc_test.py" from tofupilot.openhtf import TofuPilot test = htf.Test(phase_iqc_check) with TofuPilot(test): test.execute(test_start=lambda: input("Scan component lot: ")) ``` ## Step 3: Correlate Across Stages TofuPilot tracks results across all test stages. Open the Analytics tab to find correlations: - **Upstream predictors**: Which IQC measurements correlate with EOL failures? - **Process indicators**: Which SPI/AOI measurements predict functional test failures? - **Supplier quality**: Which vendors produce components with higher downstream failure rates? - **Time correlations**: Did failures increase after a specific process change? These correlations tell you where to add or tighten upstream detection. Each defect caught earlier is a defect that never reaches EOL or the field. ## Shift-Left Maturity Levels | Level | What You Do | Defect Detection | |-------|------------|-----------------| | 1. EOL only | All testing at end-of-line | Late, expensive | | 2. Multi-stage | IQC + FCT + EOL, data in separate systems | Earlier, but no correlation | | 3. Connected | All stages in one platform, serial-level traceability | Can correlate upstream and downstream | | 4. Predictive | ML models predict downstream failures from upstream data | Earliest, cheapest | Most companies are at level 1 or 2. The jump from 2 to 3 (connecting data across stages) is where shift-left quality becomes actionable. Level 4 (predictive) is the long-term goal. ## Common Mistakes | Mistake | Why It Fails | |---------|-------------| | Adding tests without data analysis | You add cost without knowing if the test catches real defects | | Over-testing at every stage | Cycle time and cost increase without proportional quality gain | | Shifting left without tracking results | No way to prove the upstream test is catching defects | | Removing EOL tests too early | Until upstream detection is proven, keep the safety net | Shift-left quality is data-driven. Add upstream detection, measure its effectiveness with downstream data, and iterate. The goal is catching defects at the cheapest point, not testing everything everywhere. ### Migrate from LabVIEW to Python URL: https://www.tofupilot.com/guides/how-to-migrate-from-labview-to-python-for-manufacturing-tests-with-tofupilot A practical guide to replacing LabVIEW with Python for manufacturing test automation, with concept mappings, code examples, and TofuPilot integration. LabVIEW is powerful, but it comes with $3,000-5,000/seat licensing, Windows-only deployment, and binary files that don't version-control well. Python gives you the same instrument control capabilities with zero license cost, cross-platform support, and native Git integration. This guide maps LabVIEW concepts to Python equivalents and shows you how to rebuild your test system with OpenHTF and TofuPilot. ## Why Teams Migrate | Pain Point | LabVIEW | Python | |-----------|---------|--------| | License cost | $3,160-4,840/seat/year (depending on edition) | Free | | Deployment | Windows only | Linux, macOS, Windows | | Version control | Binary .vi files, merge conflicts | Text files, Git-native | | Hiring | LabVIEW developers are scarce and expensive | Python developers are everywhere | | CI/CD | Difficult to integrate | Native (pytest, GitHub Actions, etc.) | | Package ecosystem | NI packages + limited community | pip, PyPI, 400K+ packages | | Code review | Requires LabVIEW to view | Any text editor | | Collaboration | One person per VI at a time | Standard Git workflow | ## LabVIEW to Python Concept Map | LabVIEW Concept | Python Equivalent | Notes | |----------------|-------------------|-------| | VI (Virtual Instrument) | Python function | Same idea: reusable, callable unit | | SubVI | Function or class method | Import from a module | | Front panel | No equivalent (or: TofuPilot dashboard) | Python doesn't need a GUI per function | | Block diagram | Python code | Text instead of wires | | Connector pane | Function signature | `def measure_voltage(channel: int) -> float` | | Error cluster | Exception handling | `try/except` instead of error wires | | Typedef | Class or dataclass | `@dataclass` for structured data | | Global variable | Module-level variable or class attribute | Avoid when possible | | Property node | Property decorator | `@property` on a class | | State machine | Class with methods or match/case | See example below | | TDMS file | JSON, CSV, or TofuPilot | TofuPilot replaces file-based logging | | DAQmx driver | PyVISA + pyvisa-py | Or nidaqmx Python package | | Instrument driver | OpenHTF Plug | `BasePlug` with `setUp`/`tearDown` | | Test sequence | OpenHTF Test | `htf.Test(phase1, phase2, ...)` | ## Step 1: Set Up Your Python Environment ```bash filename="migration/setup.sh" python -m venv venv source venv/bin/activate # Linux/macOS # venv\Scripts\activate # Windows pip install openhtf tofupilot pyvisa pyvisa-py ``` This gives you: - **openhtf**: Test framework (replaces LabVIEW test sequencer) - **tofupilot**: Cloud analytics (replaces TDMS file logging) - **pyvisa**: Instrument control (replaces LabVIEW instrument drivers) ## Step 2: Convert SubVIs to Python Functions A LabVIEW SubVI that reads voltage from a DMM becomes a Python function: ```python filename="migration/instrument_functions.py" def measure_voltage(channel: int) -> float: """Equivalent of a LabVIEW SubVI that reads voltage from a DMM. In LabVIEW: SubVI with channel input (I32) and voltage output (DBL). In Python: function with type hints. """ readings = {1: 3.31, 2: 5.02, 3: 1.81} # Replace with real reads return readings.get(channel, 0.0) # Call it like you'd wire a SubVI rail_3v3 = measure_voltage(1) rail_5v0 = measure_voltage(2) ``` The function signature replaces the connector pane. Type hints (`int`, `float`) replace the LabVIEW data types. ## Step 3: Replace the Error Cluster LabVIEW uses an error cluster (status, code, source) wired through every VI. Python uses exceptions. ```python filename="migration/error_handling.py" class InstrumentError(Exception): """Replaces LabVIEW error cluster for instrument errors.""" def __init__(self, code: int, source: str, message: str): self.code = code self.source = source super().__init__(f"[{code}] {source}: {message}") # LabVIEW: check error cluster at every node # Python: try/except catches errors from anywhere in the block try: voltage = measure_voltage(1) if voltage < 0: raise InstrumentError(1001, "DMM", "Negative voltage reading") except InstrumentError as e: print(f"Instrument error: {e}") # Handle or re-raise ``` This is cleaner than wiring error clusters through every node. Exceptions propagate automatically until caught. ## Step 4: Replace the State Machine LabVIEW state machines use a while loop + case structure. Python has several options. ```python filename="migration/state_machine.py" class TestSequencer: """Replaces a LabVIEW state machine for test sequencing.""" def __init__(self): self.results = {} self.state = "init" def run_step(self, name: str, func, limits: tuple) -> bool: """Run a test step and record the result.""" value = func() passed = limits[0] <= value <= limits[1] self.results[name] = { "value": value, "limits": limits, "passed": passed, } return passed def run(self): """Execute the full test sequence.""" self.run_step("rail_3v3", lambda: 3.31, (3.2, 3.4)) self.run_step("rail_5v0", lambda: 5.02, (4.8, 5.2)) all_passed = all(r["passed"] for r in self.results.values()) return all_passed seq = TestSequencer() passed = seq.run() print(f"Test {'PASSED' if passed else 'FAILED'}") ``` But you don't need to build this yourself. OpenHTF handles sequencing, measurements, and limits natively. ## Step 5: Use OpenHTF Instead of Building a Sequencer OpenHTF replaces both the LabVIEW test sequencer and the data logging (TDMS). Instrument drivers become Plugs. ```python filename="migration/openhtf_test.py" import openhtf as htf from openhtf.plugs import BasePlug from openhtf.util import units from tofupilot.openhtf import TofuPilot class InstrumentPlug(BasePlug): """Replaces LabVIEW instrument driver VIs. setUp = equivalent of opening a VISA session in LabVIEW methods = equivalent of SubVIs for each measurement tearDown = equivalent of closing the session """ def setUp(self): self._voltages = iter([3.31, 5.02]) # Replace with: rm = pyvisa.ResourceManager("@py") # self.instr = rm.open_resource("TCPIP::192.168.1.100::INSTR") def read_voltage(self) -> float: return next(self._voltages) # Replace with: return float(self.instr.query(":MEAS:VOLT:DC?")) def tearDown(self): pass # Replace with: self.instr.close() @htf.measures( htf.Measurement("rail_3v3") .in_range(3.2, 3.4) .with_units(units.VOLT) .doc("3.3V rail"), htf.Measurement("rail_5v0") .in_range(4.8, 5.2) .with_units(units.VOLT) .doc("5.0V rail"), ) @htf.plug(instr=InstrumentPlug) def test_power_rails(test, instr): """Replaces a LabVIEW test sequence with Numeric Limit Tests.""" test.measurements.rail_3v3 = instr.read_voltage() test.measurements.rail_5v0 = instr.read_voltage() def main(): test = htf.Test( test_power_rails, procedure_id="FCT-001", part_number="PCBA-100", ) with TofuPilot(test): test.execute(test_start=lambda: input("Scan serial number: ")) if __name__ == "__main__": main() ``` ## Step 6: Replace TDMS with TofuPilot LabVIEW writes test data to TDMS files. You then need to build your own analytics tools to read them. TofuPilot replaces this entire pipeline. | LabVIEW (TDMS) | TofuPilot | |----------------|-----------| | Write TDMS file after each test | Automatic upload (one line of code) | | Build custom analytics tools | Dashboard with FPY, Cpk, control charts | | Manual data aggregation across stations | Automatic multi-station aggregation | | File server for TDMS storage | Cloud storage with API access | | Custom report generation | Built-in reports and exports | ## Migration Checklist | Step | Action | LabVIEW Equivalent | |------|--------|-------------------| | 1 | Install Python + venv | Install LabVIEW | | 2 | `pip install openhtf tofupilot pyvisa` | Install NI packages | | 3 | Create Plug classes for each instrument | Create instrument driver VIs | | 4 | Write phase functions with `@htf.measures` | Create test sequence with limit checks | | 5 | Assemble `htf.Test()` with all phases | Build test sequence in LabVIEW | | 6 | Add `with TofuPilot(test)` | Configure TDMS logging | | 7 | Run and validate results match LabVIEW | Compare measurement values | ## Common Gotchas | Gotcha | Fix | |--------|-----| | "I miss the front panel" | Use TofuPilot dashboard for real-time monitoring. For operator UI, OpenHTF has a built-in web interface. | | "My NI hardware needs NI drivers" | Use the nidaqmx Python package for DAQmx hardware. For GPIB, install NI-VISA runtime. | | "LabVIEW handles threading automatically" | Python has threading and asyncio. OpenHTF handles phase execution threading. | | "My team doesn't know Python" | Python has a gentler learning curve than LabVIEW. Most engineers learn it in days, not weeks. | | "We have years of LabVIEW code" | Migrate incrementally. Start with new tests in Python. Convert existing tests one at a time. | ### ICT vs FCT vs Flying Probe: When to Use Each URL: https://www.tofupilot.com/guides/ict-vs-fct-vs-flying-probe-when-to-use-each-pcba-test-method Compare in-circuit test (ICT), functional test (FCT), and flying probe for PCBA manufacturing, with cost analysis, coverage tradeoffs, and decision criteria. Three test methods dominate PCBA manufacturing: in-circuit test (ICT), functional test (FCT), and flying probe. Each catches different defects at different costs. Most production lines use two or three in combination. This guide compares them so you can choose the right test strategy for your product. ## Quick Comparison | | ICT | FCT | Flying Probe | |--|-----|-----|-------------| | **What it tests** | Individual components | Board-level behavior | Individual components | | **Defects caught** | Wrong value, missing, short, open | Power, comms, firmware, performance | Same as ICT | | **Test time** | 5-30 seconds | 30 seconds - 5 minutes | 1-10 minutes | | **Fixture cost** | $5,000-50,000 | $1,000-10,000 | None (fixtureless) | | **NRE time** | 4-8 weeks | 1-2 weeks | Hours (program only) | | **Volume sweet spot** | 10K+ units/year | Any volume | Prototypes, < 5K units/year | | **Automation** | Fully automated | Fully automated | Fully automated | | **Board access** | Bed-of-nails (needs test pads) | Connectors + probes | Probes (no fixture) | | **Programming** | Vendor software | Python/LabVIEW/custom | Vendor software | ## In-Circuit Test (ICT) ICT presses a bed-of-nails fixture against the board and tests each component individually. It verifies that the right components are in the right places with the right values. ### What ICT Catches | Defect | Detection Method | |--------|-----------------| | Wrong resistor value | Resistance measurement | | Missing component | Open circuit detection | | Solder short | Short circuit detection | | Wrong polarity (capacitor, diode) | Capacitance/diode test | | IC pin opens | Boundary scan (JTAG) | | BGA connection issues | Limited (only accessible pins) | ### ICT Costs | Item | Typical Cost | |------|-------------| | Fixture (bed-of-nails) | $5,000-50,000 depending on board complexity | | Fixture lead time | 4-8 weeks | | ICT machine | $50,000-500,000 (capital) | | Test time per board | 5-30 seconds | | Program development | $2,000-10,000 | ### When to Use ICT - **High volume (10K+ boards/year).** The fixture cost amortizes over volume. - **Complex boards with many passives.** ICT excels at verifying resistor/capacitor values. - **SMT assembly with known defect rates.** ICT catches assembly defects before FCT. - **When you need fast test time.** 5-30 seconds vs. minutes for FCT. ### When to Skip ICT - **Low volume (< 5K/year).** Fixture cost doesn't justify the volume. - **Simple boards (< 50 components).** FCT alone catches most defects. - **Fast design iteration.** Fixture changes with every board revision. - **BGA-heavy designs.** ICT can't access BGA pads without boundary scan. ## Functional Test (FCT) FCT tests the board as a working system. You power it up, run the firmware, and verify behavior. FCT catches defects that ICT misses: firmware bugs, timing issues, analog performance, and system-level interactions. ### What FCT Catches | Defect | Detection Method | |--------|-----------------| | Voltage regulator failure | Power rail measurement | | Wrong firmware | Version query | | Communication failure | UART/SPI/I2C response check | | Excessive current draw (short) | Current measurement | | Wrong crystal frequency | Frequency measurement | | ADC/DAC calibration drift | Analog measurement | | Failed self-test | Firmware self-test command | ### FCT Costs | Item | Typical Cost | |------|-------------| | Test fixture (pogo pins + connectors) | $1,000-10,000 | | Fixture lead time | 1-2 weeks | | Instruments (DMM, PSU, etc.) | $2,000-20,000 (one-time, reusable) | | Test time per board | 30 seconds - 5 minutes | | Script development | Python + OpenHTF (free tools) | ### When to Use FCT - **Every production board should get FCT.** It's the final check before shipping. - **Any volume.** Low fixture cost makes it viable even for 100 units/year. - **After ICT.** FCT verifies system behavior that ICT can't test. - **As the only test for simple boards.** If the board has < 50 components, FCT alone may suffice. ### FCT with Python and TofuPilot ```python filename="ict_vs_fct/fct_example.py" import openhtf as htf from openhtf.plugs import BasePlug from openhtf.util import units from tofupilot.openhtf import TofuPilot class BoardPlug(BasePlug): """Board interface for FCT.""" def setUp(self): self._voltages = iter([3.31, 5.01, 1.81]) self._current = 0.12 def read_voltage(self) -> float: return next(self._voltages) def read_current(self) -> float: return self._current def query_firmware(self) -> str: return "2.1.0" def tearDown(self): pass @htf.measures( htf.Measurement("rail_3v3").in_range(3.2, 3.4).with_units(units.VOLT), htf.Measurement("rail_5v0").in_range(4.8, 5.2).with_units(units.VOLT), htf.Measurement("rail_1v8").in_range(1.7, 1.9).with_units(units.VOLT), ) @htf.plug(board=BoardPlug) def test_power(test, board): test.measurements.rail_3v3 = board.read_voltage() test.measurements.rail_5v0 = board.read_voltage() test.measurements.rail_1v8 = board.read_voltage() @htf.measures( htf.Measurement("firmware_version").equals("2.1.0"), htf.Measurement("idle_current").in_range(0.05, 0.20).with_units(units.AMPERE), ) @htf.plug(board=BoardPlug) def test_system(test, board): test.measurements.firmware_version = board.query_firmware() test.measurements.idle_current = board.read_current() def main(): test = htf.Test( test_power, test_system, procedure_id="FCT-001", part_number="PCBA-100", ) with TofuPilot(test): test.execute(test_start=lambda: input("Scan serial number: ")) ``` Every measurement flows into TofuPilot with limits, units, and pass/fail status. You get FPY, Cpk, and failure Pareto automatically. ## Flying Probe Flying probe machines use motorized probes that move to each test point. No fixture needed. The machine programs from your Gerber files and netlist. ### What Flying Probe Catches Same defects as ICT: shorts, opens, wrong values, missing components. Some machines add: - Capacitance measurement - Inductance measurement - Boundary scan (JTAG) integration - Basic functional tests (limited) ### Flying Probe Costs | Item | Typical Cost | |------|-------------| | Fixture | $0 (fixtureless) | | Setup time | Hours (from Gerber + netlist) | | Machine | $100,000-500,000 (capital) | | Test time per board | 1-10 minutes (depends on test points) | | Per-board cost at CM | $5-50 depending on program complexity | ### When to Use Flying Probe - **Prototypes and first articles.** Zero fixture cost, fast setup. - **Low volume (< 5K/year).** Cheaper than ICT fixtures. - **Design iteration.** No fixture to modify when the board changes. - **Complex BGAs.** Some flying probe machines can test BGA pads via boundary scan. ### When to Skip Flying Probe - **High volume.** Too slow (1-10 minutes vs. 5-30 seconds for ICT). - **When you need FCT anyway.** Flying probe doesn't replace FCT for system-level tests. - **Budget constraints at CM.** Some contract manufacturers charge premium for flying probe time. ## Test Strategy by Volume | Volume (units/year) | Recommended Strategy | Why | |---------------------|---------------------|-----| | 1-100 (prototype) | FCT only | Low volume, simple fixture, catches most defects | | 100-1,000 | Flying probe + FCT | Flying probe catches assembly defects, FCT catches system issues | | 1,000-10,000 | Flying probe or ICT + FCT | Evaluate ICT fixture ROI at upper end | | 10,000-100,000 | ICT + FCT | ICT fixture pays for itself, fast test time | | 100,000+ | AOI + ICT + FCT | Full coverage, maximum throughput | ## Test Strategy by Board Complexity | Board Complexity | Components | Recommended Tests | |-----------------|------------|-------------------| | Simple (LED driver, sensor board) | < 50 | FCT only | | Medium (MCU board, IoT device) | 50-200 | Flying probe + FCT | | Complex (multi-rail, mixed signal) | 200-500 | ICT + FCT | | Very complex (RF, high-speed digital) | 500+ | AOI + ICT + FCT + specialized | ## Coverage Comparison | Defect Type | AOI | ICT | Flying Probe | FCT | |------------|-----|-----|-------------|-----| | Missing component | Yes | Yes | Yes | Sometimes | | Wrong value | No | Yes | Yes | Sometimes | | Solder short | Yes | Yes | Yes | Sometimes | | Cold solder joint | Yes | No | No | Sometimes | | Tombstoning | Yes | No | No | No | | Wrong polarity | No | Yes | Yes | Sometimes | | Firmware bug | No | No | No | Yes | | Power rail issue | No | No | No | Yes | | Communication failure | No | No | No | Yes | | Performance degradation | No | No | No | Yes | No single test method catches everything. The combination of ICT (or flying probe) for assembly defects plus FCT for system defects gives you the best coverage. ### Test Data Management for Electronics URL: https://www.tofupilot.com/guides/test-data-management-for-electronics-a-developers-guide Learn how to structure, store, and query electronics test data with TofuPilot for automatic traceability, yield tracking, and process control. Every hardware test produces data. Voltages, pass/fail results, serial numbers, timestamps, operator IDs. The question isn't whether you have data. It's whether you can find it, trust it, and act on it six months later. This guide covers how to structure, store, and query electronics test data with TofuPilot so you get automatic traceability, yield tracking, and process control without building any of it yourself. ## What Test Data Management Means Test data management is the practice of organizing test results so they're queryable, traceable, and useful over time. Here's what it needs to answer: | Question | What it requires | |----------|-----------------| | Did this unit pass? | A test run linked to a serial number with a clear pass/fail outcome | | What failed and why? | Per-step measurements with limits, not just a top-level verdict | | Is yield dropping? | Time-series aggregation of pass/fail across runs | | Is this measurement drifting? | Historical measurement values with timestamps and limits | | Can we trace this unit's full history? | Every test run linked to a unit, across stations and revisions | | Which station has the worst yield? | Station-level metadata on every run | If your current system can't answer all six, you've got a data problem, not a test problem. ## Why Spreadsheets and File Systems Fail at Scale Most teams start with CSV exports or shared drives. That works for 10 units. It falls apart at 1,000. | Capability | Spreadsheets / file system | Structured database (TofuPilot) | |------------|---------------------------|-------------------------------| | Schema consistency | No. Columns drift across files | Yes. Every run follows the same model | | Query by serial number | Manual search | Instant lookup | | Yield over time | Build it yourself in Excel | Built-in, automatic | | Measurement traceability | Fragile, depends on naming | Enforced by data model | | Multi-station aggregation | Copy-paste across files | Automatic, per-station metadata | | Concurrent access | File locks, merge conflicts | Native multi-user | | Audit trail | None | Immutable run history | The core issue: flat files don't enforce relationships between units, runs, steps, and measurements. Without those relationships, every query is a one-off script. ## The Test Data Model TofuPilot organizes test data into a hierarchy that maps directly to how hardware testing works: | Entity | What it represents | Example | |--------|-------------------|---------| | Procedure | A test definition (what you're testing) | "FCT_PowerBoard_v2" | | Run | A single execution of a procedure against a unit | Run #4821, serial SN-0042, PASS | | Step | A phase or stage within a run | "measure_3v3_rail" | | Measurement | A single data point within a step, with optional limits | 3.28 V (min: 3.13, max: 3.47) | | Unit | A physical device identified by serial number | SN-0042 | | Sub-unit | A component tracked within a parent unit | WiFi module WF-1122 inside SN-0042 | Every run links to a procedure, a unit, and optionally a station. Steps and measurements nest inside runs. This structure means you can query in any direction: "show me all runs for this unit," "show me all measurements for this step across 10,000 runs," or "show me yield by station for this procedure." ## How TofuPilot Structures Test Data Automatically If you're using OpenHTF, TofuPilot captures the full test structure (phases, measurements, limits, attachments) with zero extra code. Just wrap your test with TofuPilot: ```python filename="fct_power_board.py" import openhtf as htf from openhtf.util import units from tofupilot.openhtf import TofuPilot @htf.measures( htf.Measurement("rail_3v3").in_range(3.13, 3.47).with_units(units.VOLT), ) def measure_3v3_rail(test): voltage = read_voltage("3V3_RAIL") test.measurements.rail_3v3 = voltage @htf.measures( htf.Measurement("rail_5v").in_range(4.75, 5.25).with_units(units.VOLT), ) def measure_5v_rail(test): voltage = read_voltage("5V_RAIL") test.measurements.rail_5v = voltage @htf.measures( htf.Measurement("current_draw").in_range(0.05, 0.5).with_units(units.AMPERE), ) def measure_current_draw(test): current = read_current("VIN") test.measurements.current_draw = current def main(): test = htf.Test( measure_3v3_rail, measure_5v_rail, measure_current_draw, procedure_id="FCT_PowerBoard_v2", ) with TofuPilot(test): test.execute(test_start=lambda: "SN-0042") if __name__ == "__main__": main() ``` That single `TofuPilot(test)` wrapper sends the full run (phases, measurements, limits, units, outcome, duration) to TofuPilot. No serialization code, no API calls, no file management. ## Using the Python Client Directly Not using OpenHTF? The TofuPilot Python client lets you log structured test data from any test framework or custom script: ```python filename="log_run.py" from tofupilot import TofuPilotClient client = TofuPilotClient() client.create_run( procedure_id="FCT_PowerBoard_v2", unit_under_test={"serial_number": "SN-0042"}, run_passed=True, steps=[ { "name": "measure_3v3_rail", "step_passed": True, "measurements": [ { "name": "rail_3v3", "measured_value": 3.28, "units": "V", "lower_limit": 3.13, "upper_limit": 3.47, } ], }, { "name": "measure_5v_rail", "step_passed": True, "measurements": [ { "name": "rail_5v", "measured_value": 5.01, "units": "V", "lower_limit": 4.75, "upper_limit": 5.25, } ], }, { "name": "measure_current_draw", "step_passed": True, "measurements": [ { "name": "current_draw", "measured_value": 0.12, "units": "A", "lower_limit": 0.05, "upper_limit": 0.5, } ], }, ], ) ``` Same data model, same queryability. The client handles validation, batching, and retries. ## Built-in Analytics TofuPilot computes FPY, Cpk, and failure Pareto charts automatically from your test data. Open the Analytics tab on any procedure to see yield trends, or create a custom Report for cross-procedure analysis. **First Pass Yield (FPY)** shows the percentage of units that pass on the first attempt, plotted over time. You can filter by station, date range, or unit revision. A sudden FPY drop tells you something changed: a new component lot, a fixture problem, or a test script regression. **Process Capability (Cpk)** is calculated per measurement across all runs. TofuPilot plots the measurement distribution against your spec limits and computes Cp and Cpk automatically. A Cpk below 1.33 means your process is too close to the limits. **Failure Pareto** ranks test steps by failure count so you can focus on the biggest contributor first. TofuPilot builds this chart for any procedure, any time range. **Custom Reports** let you combine FPY, Cpk, and failure data across multiple procedures into a single view. Use these for weekly quality reviews or to compare yield across production lines. ## Comparison: Approaches to Test Data Management | Capability | CSV / shared drive | Custom database | TofuPilot | |------------|-------------------|----------------|-----------| | Structured data model | No | You build it | Built in | | Query by serial number | Manual | SQL queries | Instant search | | FPY tracking | Spreadsheet formulas | You build it | Automatic | | Cpk analysis | Export to Minitab | You build it | Built in per measurement | | Failure Pareto | Manual sorting | You build it | Automatic | | Multi-station support | Separate files | You build it | Native, per-station metadata | | Unit traceability | Fragile | You build it | Full history per serial number | | Sub-unit tracking | Not practical | You build it | Built in | | Audit trail | None | You build it | Immutable | | Setup time | Minutes | Weeks to months | Minutes | | Maintenance | Low (until it breaks) | Ongoing | Zero | The pattern is clear. You can build all of this yourself, and many teams have. But every hour spent on test infrastructure is an hour not spent on the product you're actually testing. ### Semiconductor Wafer Test Data with TofuPilot URL: https://www.tofupilot.com/guides/semiconductor-wafer-test-data-with-tofupilot Learn how to track wafer-level and final test data for semiconductors using TofuPilot for CP/FT yield analysis and bin mapping. # Semiconductor Wafer Test Data with TofuPilot Semiconductor testing happens in two stages: wafer-level testing (CP, circuit probe) and final test (FT, packaged parts). Each stage generates massive datasets. TofuPilot stores per-die and per-part test results with full measurement data, enabling yield analysis, bin mapping, and lot traceability. ## Semiconductor Test Flow ``` Wafer Fab → Wafer Test (CP) → Packaging → Final Test (FT) → Binning → Ship ↓ ↓ TofuPilot TofuPilot (per-die data) (per-part data) ``` ### Wafer Test (Circuit Probe) Probes contact each die on the wafer. Tests include: | Test | What it checks | |------|---------------| | Continuity/opens | Bond pad connectivity | | Leakage | Junction and oxide integrity | | Parametric | Vth, Idsat, Rds(on), timing | | Functional | Basic digital/analog operation | ### Final Test (FT) After packaging, full parametric and functional testing at speed. | Test | What it checks | |------|---------------| | DC parametric | Input/output levels, power consumption | | AC parametric | Speed, setup/hold times, rise/fall times | | Functional | Full device operation at target frequency | | Burn-in (optional) | Infant mortality screening | ## Logging Wafer Test Data ### Per-Die Results ```python filename="wafer_test.py" from tofupilot import TofuPilotClient client = TofuPilotClient() def log_die_test(wafer_id, die_x, die_y, measurements, bin_code): """Log test results for a single die.""" die_id = f"{wafer_id}-X{die_x}Y{die_y}" steps = [{ "name": "Parametric", "step_type": "measurement", "status": bin_code == 1, # Bin 1 = good die "measurements": [ {"name": "vth_mv", "value": measurements["vth"], "unit": "mV", "limit_low": 350, "limit_high": 450}, {"name": "idsat_ua_um", "value": measurements["idsat"], "unit": "uA/um", "limit_low": 500, "limit_high": 700}, {"name": "ioff_pa_um", "value": measurements["ioff"], "unit": "pA/um", "limit_high": 100}, {"name": "rdson_mohm", "value": measurements["rdson"], "unit": "mohm", "limit_high": 50}, ], }] client.create_run( procedure_id="WAFER-CP-TEST", unit_under_test={ "serial_number": die_id, "part_number": "IC-POWER-MGMT-V3", }, run_passed=bin_code == 1, steps=steps, ) # Test all dies on a wafer wafer_id = "LOT2025A-W07" for x in range(20): for y in range(20): meas = probe_die(x, y) bin_code = classify_die(meas) log_die_test(wafer_id, x, y, meas, bin_code) ``` ### Wafer-Level Summary After testing all dies, upload a wafer summary. ```python filename="wafer_summary.py" # Wafer-level summary total_dies = 400 good_dies = 352 yield_pct = good_dies / total_dies * 100 client.create_run( procedure_id="WAFER-CP-SUMMARY", unit_under_test={ "serial_number": "LOT2025A-W07", "part_number": "IC-POWER-MGMT-V3", }, run_passed=yield_pct > 80, steps=[{ "name": "Wafer Summary", "step_type": "measurement", "status": yield_pct > 80, "measurements": [ {"name": "total_dies", "value": total_dies, "unit": "count"}, {"name": "good_dies", "value": good_dies, "unit": "count"}, {"name": "cp_yield_pct", "value": yield_pct, "unit": "%", "limit_low": 80}, {"name": "bin1_count", "value": 352, "unit": "count"}, {"name": "bin2_count", "value": 30, "unit": "count"}, {"name": "bin3_count", "value": 18, "unit": "count"}, ], }], ) ``` ## Bin Mapping Binning assigns each die or part to a category based on test results. Common bin structure: | Bin | Meaning | Typical criteria | |-----|---------|-----------------| | 1 | Good, full spec | All parametrics within spec | | 2 | Good, derated | Parametrics within wider limits | | 3 | Functional fail | Failed functional test | | 4 | Parametric fail | One or more parametric out of spec | | 5 | Continuity fail | Open or short detected | | 6 | Leakage fail | Excessive junction leakage | Track bin distributions in TofuPilot across wafers, lots, and time periods. ## Yield Analysis ### Wafer-to-Wafer Yield Compare CP yield across wafers in a lot. | Wafer | Total dies | Good (Bin 1) | Yield | |-------|-----------|--------------|-------| | W01 | 400 | 360 | 90.0% | | W02 | 400 | 355 | 88.8% | | W03 | 400 | 312 | 78.0% | | W04 | 400 | 358 | 89.5% | W03 has significantly lower yield. Investigate wafer-level process issues (particle count, lithography focus, etch uniformity). ### Lot-to-Lot Yield Track yield across production lots to monitor fab process stability. ```python filename="lot_yield_tracking.py" from tofupilot import TofuPilotClient client = TofuPilotClient() # Get all wafer summary runs runs = client.get_runs( procedure_id="WAFER-CP-SUMMARY", limit=100, ) # Group by lot and calculate average yield lot_yields = {} for run in runs: serial = run["unit_under_test"]["serial_number"] lot = serial.split("-W")[0] # Extract lot ID for step in run.get("steps", []): for m in step.get("measurements", []): if m["name"] == "cp_yield_pct": if lot not in lot_yields: lot_yields[lot] = [] lot_yields[lot].append(m["value"]) for lot, yields in sorted(lot_yields.items()): avg = sum(yields) / len(yields) print(f"{lot}: {avg:.1f}% avg yield ({len(yields)} wafers)") ``` ### CP to FT Yield Correlation Track yield at both test stages to find packaging-related issues. | Lot | CP Yield | FT Yield | FT/CP Loss | |-----|----------|----------|------------| | LOT2025A | 89.2% | 87.1% | 2.1% | | LOT2025B | 91.0% | 88.5% | 2.5% | | LOT2025C | 90.5% | 82.3% | 8.2% | LOT2025C shows excessive CP-to-FT yield loss (8.2% vs. typical 2-3%). This points to a packaging issue: wire bond failures, die attach problems, or moisture sensitivity during assembly. ## Parametric Distribution Monitoring Track critical parametric distributions across production to catch process drift. Key parameters to monitor: | Parameter | Why it matters | |-----------|---------------| | Vth (threshold voltage) | Shifts indicate gate oxide or implant dose issues | | Idsat (saturation current) | Indicates transistor drive strength | | Ioff (off-state leakage) | Affects standby power, process control indicator | | Rdson (on-resistance) | Critical for power devices | | Timing margins | Speed grading, binning into speed grades | TofuPilot's measurement histograms show the distribution of each parameter across all tested dies. A bimodal distribution suggests two different process conditions. A shifting mean suggests process drift. ## Traceability from Part to Wafer When a packaged part fails in the field, trace back: 1. Part serial number → packaging lot 2. Packaging lot → wafer ID 3. Wafer ID → die position (X, Y) 4. Die position → original CP test data All of this data lives in TofuPilot, linked by serial numbers. Was the die marginal at wafer test? Was it a Bin 2 (derated) part sold as Bin 1? The data answers these questions. ### Time-Series Test Data Analysis with TofuPilot URL: https://www.tofupilot.com/guides/time-series-test-data-analysis-with-tofupilot Learn how to capture, store, and analyze time-series measurement data from hardware tests using TofuPilot's multi-dimensional arrays. # Time-Series Test Data Analysis with TofuPilot Hardware tests produce waveforms, not just single numbers. A power supply ripple test captures thousands of voltage samples over time. A vibration test records acceleration spectra across frequency bands. TofuPilot stores these time-series measurements natively, so you can trend and compare waveforms across your production. ## Single Values vs. Time-Series Most test systems store one number per measurement: "output voltage = 3.31V." But the full story is in the waveform. That 3.31V might be a clean DC signal or a noisy mess that happens to average out to 3.31V. | Data type | Example | What it reveals | |-----------|---------|----------------| | Single value | Vout = 3.31V | Average output level | | Time series | 1000 samples over 10ms | Ripple, noise, transient behavior | | Frequency spectrum | FFT of output voltage | Switching noise frequency content | | Multi-axis | X/Y/Z acceleration | Vibration in all directions | TofuPilot handles all of these through multi-dimensional measurement arrays. ## Capturing Time-Series Data ### With OpenHTF Use dimensioned measurements to store time-series data. ```python filename="ripple_test_openhtf.py" import openhtf as htf from tofupilot.openhtf import TofuPilotClient import numpy as np @htf.measures( htf.Measurement("output_ripple_mv") .with_dimensions("time_us") .with_units("mV"), htf.Measurement("ripple_pk_pk_mv") .in_range(0, 50) .with_units("mV"), ) def ripple_test(test): # Capture 1000 samples at 100kHz (10us spacing) waveform = capture_oscilloscope(channel=1, samples=1000, rate_hz=100000) for i, sample in enumerate(waveform): test.measurements.output_ripple_mv[i * 10] = sample # time in microseconds # Also store the scalar summary test.measurements.ripple_pk_pk_mv = max(waveform) - min(waveform) def main(): test = htf.Test(ripple_test) test.add_output_callbacks(TofuPilotClient()) test.execute(lambda: "PSU-2025-0099") ``` ### With the Python Client Pass lists or arrays directly. ```python filename="ripple_test_client.py" from tofupilot import TofuPilotClient import numpy as np client = TofuPilotClient() # Capture waveform from oscilloscope waveform = capture_oscilloscope(channel=1, samples=1000, rate_hz=100000) pk_pk = float(np.max(waveform) - np.min(waveform)) client.create_run( procedure_id="PSU-RIPPLE-TEST", unit_under_test={"serial_number": "PSU-2025-0099"}, run_passed=pk_pk < 50, steps=[{ "name": "Output Ripple", "step_type": "measurement", "status": pk_pk < 50, "measurements": [ { "name": "ripple_waveform_mv", "value": waveform.tolist(), "unit": "mV", }, { "name": "ripple_pk_pk_mv", "value": pk_pk, "unit": "mV", "limit_high": 50, }, ], }], ) ``` ## Types of Time-Series Test Data ### Voltage/Current Waveforms Captured from oscilloscopes or DAQs during power-on sequences, ripple tests, or transient response tests. Store the raw waveform plus scalar summaries (peak-to-peak, RMS, rise time). ### Temperature Profiles Recorded during thermal cycling, burn-in, or heat dissipation tests. Multiple sensors produce multi-channel time-series data over minutes or hours. ```python filename="thermal_profile.py" # Thermal test with 4 temperature sensors sampled every second for 30 minutes sensors = ["junction", "ambient", "heatsink", "case"] duration_s = 1800 samples_per_sensor = duration_s # 1 sample/second for sensor_name in sensors: readings = read_thermal_sensor(sensor_name, samples=samples_per_sensor) measurements.append({ "name": f"temp_{sensor_name}", "value": readings, # list of 1800 values "unit": "°C", }) ``` ### Vibration Spectra FFT data from accelerometers during vibration testing. Frequency on one axis, amplitude on the other. ### Pressure/Flow Curves Time-series pressure and flow measurements during leak tests, pneumatic tests, or hydraulic validation. ## Analyzing Time-Series Across Production The power of storing waveforms (not just scalars) is comparison across units. ### Waveform Overlay Compare ripple waveforms from 100 units. If 99 look the same and one has an extra spike, that unit has a problem that the peak-to-peak measurement alone might not catch. ### Statistical Bounds Calculate the mean and standard deviation of your waveform at each time point across all units. This gives you an envelope of normal behavior. Any unit whose waveform falls outside the envelope is flagged. ```python filename="waveform_statistics.py" import numpy as np # All waveforms from production (each is a list of 1000 samples) all_waveforms = np.array(waveforms_from_tofupilot) # shape: (N_units, 1000) mean_waveform = np.mean(all_waveforms, axis=0) std_waveform = np.std(all_waveforms, axis=0) upper_bound = mean_waveform + 3 * std_waveform lower_bound = mean_waveform - 3 * std_waveform # Check if a new unit's waveform is within bounds new_waveform = np.array(new_unit_data) is_anomalous = np.any(new_waveform > upper_bound) or np.any(new_waveform < lower_bound) ``` ### Trend Analysis on Waveform Features Extract features from each waveform (rise time, settling time, overshoot, RMS) and trend them over production. A gradual increase in rise time across units suggests a component or process drift. ## Best Practices | Practice | Why | |----------|-----| | Store both raw waveform and scalar summaries | Waveforms for deep analysis, scalars for dashboards and trending | | Use consistent sample rates | Comparing waveforms requires the same number of points and timing | | Include time axis metadata | Sample rate or time stamps so the waveform can be reconstructed | | Set limits on scalar summaries | Scalars drive pass/fail, waveforms drive root cause analysis | | Limit waveform size | Keep arrays under 10,000 points per measurement for practical storage | ### Manage Operator Certification URL: https://www.tofupilot.com/guides/how-to-manage-operator-certification-with-tofupilot Learn how to track operator training, certification status, and test authorization using TofuPilot properties and station access controls. # How to Manage Operator Certification with TofuPilot A new operator runs a test wrong and scraps three boards before anyone notices. It happens more than it should. Most teams track operator certification in spreadsheets that go stale the day they're created. TofuPilot lets you tie operator identity to every test run and enforce certification requirements at the station level. ## Why Operator Tracking Matters Regulated industries require it. ISO 13485 (medical devices), AS9100 (aerospace), and IATF 16949 (automotive) all mandate that operators are trained and qualified for the tasks they perform. But even without regulatory pressure, knowing who ran what test matters when you're debugging a yield drop. ## Prerequisites - A TofuPilot account - Python 3.8+ with `tofupilot` installed - An operator authentication method (badge scan, login, or barcode) ## Step 1: Capture Operator Identity on Every Run The simplest approach is to record the operator as a run property: ```python filename="operator_tracking.py" from tofupilot import TofuPilotClient client = TofuPilotClient() operator_id = input("Scan operator badge: ") result = client.create_run( procedure_id="pcba-fct-v2", unit_under_test={ "serial_number": dut_serial, "part_number": "PCB-100-R4", }, run_passed=True, properties={ "operator_id": operator_id, "operator_name": get_operator_name(operator_id), "station_id": "ST-04", }, ) ``` Every run now has an operator attached. You can filter and analyze by operator in the TofuPilot dashboard. ## Step 2: Build an Operator Certification Check Before allowing a test to run, verify the operator is certified for that procedure: ```python filename="certification_check.py" import json from pathlib import Path # Certification data (could also come from an API or database) CERT_FILE = Path("operator_certs.json") def load_certifications(): """Load operator certification records.""" if CERT_FILE.exists(): return json.loads(CERT_FILE.read_text()) return {} def is_certified(operator_id: str, procedure_id: str) -> bool: """Check if operator is certified for a specific test procedure.""" certs = load_certifications() operator = certs.get(operator_id, {}) procedures = operator.get("certified_procedures", []) return procedure_id in procedures def require_certification(operator_id: str, procedure_id: str): """Block test execution if operator isn't certified.""" if not is_certified(operator_id, procedure_id): raise PermissionError( f"Operator {operator_id} is not certified for {procedure_id}. " f"Contact your line supervisor." ) ``` Example certification file: ```json filename="operator_certs.json" { "OP-001": { "name": "Jane Chen", "certified_procedures": ["pcba-fct-v2", "motor-fct", "burn-in-48h"], "certification_date": "2026-01-15", "expiry_date": "2027-01-15" }, "OP-002": { "name": "Mike Torres", "certified_procedures": ["pcba-fct-v2"], "certification_date": "2026-02-01", "expiry_date": "2027-02-01" } } ``` ## Step 3: Integrate with Your Test Sequence Wire the certification check into your test startup: ```python filename="certified_test.py" import openhtf as htf from tofupilot import TofuPilotClient from certification_check import require_certification PROCEDURE_ID = "pcba-fct-v2" def test_start(test): """Scan operator badge and verify certification before testing.""" operator_id = input("Scan operator badge: ") # Block uncertified operators require_certification(operator_id, PROCEDURE_ID) test.state["operator_id"] = operator_id test.dut_id = input("Scan DUT serial: ") def main(): test = htf.Test( functional_tests, power_tests, ) test.add_output_callbacks( TofuPilotClient().as_openhtf_callback( procedure_id=PROCEDURE_ID, ) ) test.execute(test_start=htf.PhaseDescriptor.wrap(test_start)) ``` ## Step 4: Track Certification Expiry Certifications expire. Build a simple check that warns before expiry and blocks after: ```python filename="cert_expiry.py" from datetime import date, timedelta def check_certification_status(operator_id: str, procedure_id: str): """Check certification validity with advance warning.""" certs = load_certifications() operator = certs.get(operator_id) if not operator: raise PermissionError(f"Unknown operator: {operator_id}") if procedure_id not in operator.get("certified_procedures", []): raise PermissionError( f"{operator['name']} is not certified for {procedure_id}" ) expiry = date.fromisoformat(operator["expiry_date"]) today = date.today() if today > expiry: raise PermissionError( f"Certification expired on {expiry}. Recertification required." ) days_remaining = (expiry - today).days if days_remaining < 30: print(f"WARNING: Certification expires in {days_remaining} days") return True ``` ## Step 5: Analyze Operator Performance With operator data on every run, you can answer questions that matter: - **Yield by operator**: Is one operator consistently lower? They might need retraining. - **Test duration by operator**: Slower operators may be following procedures more carefully, or struggling with the equipment. - **Failure modes by operator**: If one operator sees more of a specific failure, check their technique. ```python filename="operator_analysis.py" from tofupilot import TofuPilotClient from collections import defaultdict client = TofuPilotClient() runs = client.get_runs( procedure_id="pcba-fct-v2", limit=1000, ) # Group by operator operator_stats = defaultdict(lambda: {"pass": 0, "fail": 0}) for run in runs: op = run.properties.get("operator_id", "unknown") if run.passed: operator_stats[op]["pass"] += 1 else: operator_stats[op]["fail"] += 1 for op_id, stats in operator_stats.items(): total = stats["pass"] + stats["fail"] fpy = stats["pass"] / total * 100 print(f"Operator {op_id}: {fpy:.1f}% FPY ({total} runs)") ``` ## Regulatory Compliance Notes | Standard | Requirement | TofuPilot Solution | |---|---|---| | ISO 13485 | Documented training records, competency assessment | Operator ID on every run, certification check | | AS9100 | Personnel qualified for assigned tasks | Pre-test certification gate | | IATF 16949 | Training effectiveness evaluated | Yield-by-operator analysis | | FDA 21 CFR 820 | Personnel training documented | Full audit trail with operator identity | The key is that every test run links back to a certified operator, and that link is immutable in your test history. ### How to Add Operator Prompts to Hardware Tests URL: https://www.tofupilot.com/guides/how-to-add-operator-prompts-to-hardware-tests Operator prompts pause automated tests for manual steps. Learn how to add prompts with text input, choices, and timeouts to OpenHTF tests. # How to Add Operator Prompts to Hardware Tests Many manufacturing tests mix automated measurements with manual steps. The operator needs to load a DUT, flip a switch, perform a visual inspection, or enter a value. OpenHTF handles this through operator prompts that pause the test, display a message, and optionally collect input. This guide covers how to add prompts to your tests and display them through the TofuPilot operator UI. ## When to Use Prompts | Situation | Example | |-----------|---------| | DUT loading | "Place the board in the fixture and close the clamp" | | Visual inspection | "Check for solder bridges on connector J1" | | Manual measurement | "Read the label and enter the lot code" | | Physical action | "Press the reset button on the DUT" | | Go/no-go decision | "Does the LED illuminate green?" | | Safety check | "Verify the safety cover is closed" | Prompts keep the operator in the test flow. They see the instruction in the same interface that shows measurements and pass/fail results. ## Prerequisites - Python 3.10+ - OpenHTF installed (`pip install openhtf`) - TofuPilot Python SDK installed (`pip install tofupilot`) ## Step 1: Add a Simple Prompt The simplest prompt displays a message and waits for the operator to acknowledge it. ```python filename="prompted_test.py" import openhtf as htf from openhtf.plugs import user_input @htf.plug(prompts=user_input.UserInput) def phase_load_dut(test, prompts): """Wait for the operator to load the DUT.""" prompts.prompt( "Place the board in the test fixture. " "Close the clamp and press Enter." ) ``` The test pauses at this phase until the operator responds. In TofuPilot's operator UI, the prompt appears as a card with the message and an acknowledgment button. ## Step 2: Collect Text Input Add `text_input=True` to collect a value from the operator. The returned string can be stored as a measurement. ```python filename="prompted_test.py" @htf.plug(prompts=user_input.UserInput) @htf.measures( htf.Measurement("lot_code").with_args(docstring="Operator-entered lot code"), ) def phase_enter_lot_code(test, prompts): """Ask the operator to scan or type the lot code.""" lot = prompts.prompt( "Scan the lot code barcode on the packaging.", text_input=True, ) test.measurements.lot_code = lot ``` ## Step 3: Use Prompts for Visual Inspection Combine a prompt with a measurement to record the operator's inspection result. ```python filename="prompted_test.py" @htf.plug(prompts=user_input.UserInput) @htf.measures( htf.Measurement("visual_result").equals("PASS"), ) def phase_visual_inspection(test, prompts): """Operator checks for cosmetic defects.""" result = prompts.prompt( "Inspect the board under magnification. " "Type PASS if no defects found, or FAIL if defects are present.", text_input=True, ) test.measurements.visual_result = result.strip().upper() ``` ## Step 4: Add Prompts Between Automated Phases Mix prompted and automated phases in the same test. The operator sees a seamless flow: automated measurements run, then a prompt appears, then more automated measurements. ```python filename="prompted_test.py" from openhtf.util import units @htf.measures( htf.Measurement("supply_voltage_V") .in_range(minimum=4.9, maximum=5.1) .with_units(units.VOLT), ) def phase_power_check(test): """Automated: measure supply voltage.""" test.measurements.supply_voltage_V = 5.01 @htf.plug(prompts=user_input.UserInput) def phase_flip_board(test, prompts): """Ask operator to flip the board for bottom-side testing.""" prompts.prompt( "Flip the board over so the bottom side faces up. " "Press Enter when ready." ) @htf.measures( htf.Measurement("bottom_connector_resistance_mOhm") .in_range(maximum=100) .with_units(units.OHM), ) def phase_bottom_test(test): """Automated: measure bottom-side connector resistance.""" test.measurements.bottom_connector_resistance_mOhm = 38.5 ``` ## Step 5: Connect to TofuPilot Wire all phases together. Prompted phases render in the TofuPilot operator UI automatically. No extra configuration needed. ```python filename="prompted_test.py" from tofupilot.openhtf import TofuPilot test = htf.Test( phase_load_dut, phase_enter_lot_code, phase_power_check, phase_visual_inspection, phase_flip_board, phase_bottom_test, ) with TofuPilot(test): test.execute(test_start=lambda: input("Scan serial: ")) ``` ## Best Practices | Practice | Why | |----------|-----| | Keep prompt text short (1-2 sentences) | Operators scan, they don't read paragraphs | | Use imperative verbs ("Place", "Press", "Scan") | Clear instructions reduce errors | | One action per prompt | Don't ask the operator to do three things at once | | Validate operator input | Use `.equals()` or `.matches_regex()` on the measurement | | Put prompts at natural breaks | Between automated sequences, not in the middle of a measurement | | Avoid unnecessary prompts | Every prompt adds cycle time. Automate what you can. | ### SCPI Commands in Python for Test Engineers URL: https://www.tofupilot.com/guides/scpi-commands-in-python-a-complete-guide-for-test-engineers A reference for SCPI commands in Python using PyVISA, covering measurement types, triggering, error handling, and OpenHTF integration with TofuPilot. SCPI (Standard Commands for Programmable Instruments) is the command language that most modern test instruments speak. If your multimeter, power supply, or oscilloscope was made after 1990, it almost certainly supports SCPI. This guide covers the commands you'll use most in manufacturing test, how to send them from Python with PyVISA, and how to integrate them into production tests with OpenHTF and TofuPilot. ## What Is SCPI SCPI defines a standard set of text commands for controlling instruments. Instead of learning a different protocol for every vendor, you use the same command structure across Keysight, Rigol, Rohde & Schwarz, Tektronix, and others. Commands follow a tree structure: ``` :MEASure :VOLTage :DC? → Measure DC voltage :AC? → Measure AC voltage :CURRent :DC? → Measure DC current :RESistance? → Measure resistance ``` The colon separates levels. A question mark means "query" (read a value). No question mark means "command" (set something). ## Setup ```bash filename="install.sh" pip install pyvisa pyvisa-py ``` Connect to an instrument: ```python filename="scpi/connect.py" import pyvisa rm = pyvisa.ResourceManager("@py") # Pure Python backend dmm = rm.open_resource("TCPIP::192.168.1.100::INSTR") dmm.timeout = 5000 # 5 seconds # Every SCPI instrument responds to *IDN? idn = dmm.query("*IDN?") print(f"Connected: {idn.strip()}") ``` ## Common SCPI Commands ### IEEE 488.2 Mandatory Commands Every SCPI instrument supports these: | Command | Purpose | Returns | |---------|---------|---------| | `*IDN?` | Identify instrument | Manufacturer, model, serial, firmware | | `*RST` | Reset to factory defaults | Nothing | | `*CLS` | Clear status and error queue | Nothing | | `*OPC?` | Operation complete query | "1" when done | | `*TST?` | Self-test | 0 = pass, nonzero = fail | | `*WAI` | Wait for pending operations | Nothing (blocks until done) | Always start a test with `*RST` and `*CLS`. This puts the instrument in a known state. ```python filename="scpi/reset.py" dmm.write("*RST") dmm.write("*CLS") ``` ### Measurement Commands Two patterns: shorthand and explicit. **Shorthand** (configure + trigger + read in one command): ```python filename="scpi/measure_shorthand.py" # DC voltage, auto-range voltage = float(dmm.query(":MEAS:VOLT:DC?")) # DC current, auto-range current = float(dmm.query(":MEAS:CURR:DC?")) # Resistance, auto-range resistance = float(dmm.query(":MEAS:RES?")) # AC voltage ac_voltage = float(dmm.query(":MEAS:VOLT:AC?")) ``` **Explicit** (separate configure, trigger, read for more control): ```python filename="scpi/measure_explicit.py" # Configure for DC voltage, 10V range dmm.write(":CONF:VOLT:DC 10") # Trigger a measurement dmm.write(":INIT") # Wait for completion dmm.query("*OPC?") # Read the result voltage = float(dmm.query(":FETCH?")) ``` Use the explicit pattern when you need precise timing control or when measuring multiple channels in sequence. | Pattern | When to Use | Commands | |---------|------------|----------| | `:MEAS:...?` | Quick single reading | 1 command | | `:CONF:` + `:INIT` + `:FETCH?` | Precise timing, multi-channel | 3+ commands | | `:CONF:` + `:READ?` | Configure once, read repeatedly | 2 commands | ### Power Supply Commands ```python filename="scpi/power_supply.py" import pyvisa import time rm = pyvisa.ResourceManager("@py") psu = rm.open_resource("TCPIP::192.168.1.101::INSTR") psu.timeout = 5000 psu.write("*RST") psu.write("*CLS") # Select channel and configure psu.write(":INST:SEL CH1") psu.write(":VOLT 5.0") # Set 5V output psu.write(":CURR 0.5") # 500mA current limit # Enable output psu.write(":OUTP ON") time.sleep(0.5) # Wait for stabilization # Read actual values actual_v = float(psu.query(":MEAS:VOLT?")) actual_i = float(psu.query(":MEAS:CURR?")) print(f"Output: {actual_v:.3f}V, {actual_i:.4f}A") # Always disable output at the end psu.write(":OUTP OFF") psu.close() ``` Common power supply commands: | Command | Purpose | |---------|---------| | `:INST:SEL CH1` | Select output channel | | `:VOLT 5.0` | Set voltage | | `:CURR 0.5` | Set current limit | | `:OUTP ON` / `:OUTP OFF` | Enable/disable output | | `:MEAS:VOLT?` | Read actual output voltage | | `:MEAS:CURR?` | Read actual output current | | `:VOLT:PROT 6.0` | Set over-voltage protection | | `:CURR:PROT 1.0` | Set over-current protection | ### Oscilloscope Commands ```python filename="scpi/oscilloscope.py" # Basic oscilloscope setup scope = rm.open_resource("TCPIP::192.168.1.102::INSTR") scope.timeout = 10000 scope.write("*RST") scope.write(":CHAN1:DISP ON") # Enable channel 1 scope.write(":CHAN1:SCAL 1.0") # 1V/div scope.write(":TIM:SCAL 0.001") # 1ms/div scope.write(":TRIG:EDGE:SOUR CHAN1") # Trigger on channel 1 scope.write(":TRIG:EDGE:LEV 1.5") # Trigger at 1.5V # Measure frequency and amplitude frequency = float(scope.query(":MEAS:FREQ? CHAN1")) amplitude = float(scope.query(":MEAS:VAMP? CHAN1")) ``` ## Error Handling Instruments report errors through a queue. Always drain the queue after a command sequence. ```python filename="scpi/error_handling.py" def check_errors(instr) -> list[str]: """Read all errors from the SCPI error queue.""" errors = [] while True: err = instr.query(":SYST:ERR?").strip() code = int(err.split(",")[0]) if code == 0: break errors.append(err) return errors # After a sequence of commands errors = check_errors(dmm) if errors: for e in errors: print(f" Error: {e}") raise RuntimeError(f"Instrument errors: {errors}") ``` Common SCPI error codes: | Code | Meaning | Typical Cause | |------|---------|---------------| | -100 | Command error | Typo in command string | | -200 | Execution error | Invalid parameter value | | -300 | Device-specific error | Hardware issue | | -400 | Query error | Query without reading response | | 0 | No error | Queue empty | ## Triggering Models SCPI defines several trigger sources. The right choice depends on your timing requirements. | Trigger Source | Command | Use Case | |---------------|---------|----------| | Immediate | `:TRIG:SOUR IMM` | Default. Measures as soon as configured. | | Bus | `:TRIG:SOUR BUS` | Software trigger with `*TRG`. Precise timing. | | External | `:TRIG:SOUR EXT` | Hardware trigger line. Synchronized with other instruments. | | Timer | `:TRIG:SOUR TIM` | Periodic measurements at fixed intervals. | Bus trigger example (precise timing control): ```python filename="scpi/bus_trigger.py" dmm.write(":CONF:VOLT:DC 10") # Configure range dmm.write(":TRIG:SOUR BUS") # Bus trigger mode dmm.write(":INIT") # Arm the trigger system # ... do other setup ... dmm.write("*TRG") # Send the trigger dmm.query("*OPC?") # Wait for completion reading = float(dmm.query(":FETCH?")) ``` ## Vendor-Specific Differences SCPI is a standard, but vendors add their own extensions. Common differences: | Feature | Keysight | Rigol | Rohde & Schwarz | |---------|----------|-------|-----------------| | Channel select | `:INST:SEL CH1` | `:INST CH1` | `:INST:SEL 1` | | Screenshot | `:DISP:DATA? PNG` | `:DISP:DATA?` | `:HCOP:DATA?` | | Error query | `:SYST:ERR?` | `:SYST:ERR?` | `:SYST:ERR:ALL?` | | Beeper | `:SYST:BEEP` | `:SYST:BEEP:STAT ON` | `:SYST:BEEP:IMM` | Always check your instrument's programming manual for exact syntax. The core measurement commands (`:MEAS:`, `:CONF:`, `:INIT`, `:FETCH?`) are consistent across vendors. ## Integration with OpenHTF and TofuPilot Wrap SCPI commands in an OpenHTF Plug for production tests. The plug handles connection lifecycle, and measurements flow through OpenHTF into TofuPilot automatically. ```python filename="scpi/openhtf_integration.py" import pyvisa import openhtf as htf from openhtf.plugs import BasePlug from openhtf.util import units from tofupilot.openhtf import TofuPilot class ScpiDmm(BasePlug): """SCPI multimeter plug with automatic lifecycle.""" RESOURCE = "TCPIP::192.168.1.100::INSTR" def setUp(self): rm = pyvisa.ResourceManager("@py") self.instr = rm.open_resource(self.RESOURCE) self.instr.timeout = 5000 self.instr.write("*RST") self.instr.write("*CLS") def measure_dc_voltage(self, range_v: str = "AUTO") -> float: self.instr.write(f":CONF:VOLT:DC {range_v}") return float(self.instr.query(":MEAS:VOLT:DC?")) def measure_dc_current(self, range_a: str = "AUTO") -> float: self.instr.write(f":CONF:CURR:DC {range_a}") return float(self.instr.query(":MEAS:CURR:DC?")) def measure_resistance(self, range_ohm: str = "AUTO") -> float: self.instr.write(f":CONF:RES {range_ohm}") return float(self.instr.query(":MEAS:RES?")) def tearDown(self): self.instr.close() @htf.measures( htf.Measurement("voltage_3v3") .in_range(3.2, 3.4) .with_units(units.VOLT), ) @htf.plug(dmm=ScpiDmm) def test_rail(test, dmm): test.measurements.voltage_3v3 = dmm.measure_dc_voltage() def main(): test = htf.Test(test_rail, procedure_id="FCT-001", part_number="PCBA-100") with TofuPilot(test): test.execute(test_start=lambda: input("Serial: ")) ``` ## Quick Reference | Task | SCPI Command | |------|-------------| | Identify | `*IDN?` | | Reset | `*RST` | | Clear errors | `*CLS` | | Measure DC voltage | `:MEAS:VOLT:DC?` | | Measure DC current | `:MEAS:CURR:DC?` | | Measure resistance | `:MEAS:RES?` | | Set voltage (PSU) | `:VOLT 5.0` | | Set current limit (PSU) | `:CURR 0.5` | | Output on/off (PSU) | `:OUTP ON` / `:OUTP OFF` | | Read error | `:SYST:ERR?` | | Wait for complete | `*OPC?` | | Software trigger | `*TRG` | ### Track Firmware Versions in Production URL: https://www.tofupilot.com/guides/how-to-track-firmware-versions-in-production-with-tofupilot Record firmware versions as metadata in OpenHTF tests so TofuPilot lets you filter runs, trace failures, and prove compliance by software build. When a unit fails in the field, the first question is "what firmware was it running?" If you can't answer that from your test records, you're stuck guessing. TofuPilot lets you tag every test run with the firmware version so you can filter, trace, and report on it later. ## Why Firmware Version Tracking Matters Firmware changes between production batches are common. A bug fix in v2.4.2 might introduce a regression that only shows up under specific conditions. Without version tracking, you can't correlate failure spikes with firmware releases. Regulatory standards (IEC 62304, FDA 21 CFR Part 820, ISO 13485) require you to document the software configuration of each device at the time of production. For field returns, you need to prove which firmware was loaded when the unit left your factory. Firmware tracking also helps during EVT/DVT/PVT transitions. You can compare FPY and measurement distributions across firmware versions to verify that a new build doesn't degrade performance. ## Recording Firmware Version as a Measurement The simplest approach is to record the firmware version as a measurement in your test. This makes it visible in every run record. ```python filename="firmware_metadata_test.py" import openhtf as htf from tofupilot.openhtf import TofuPilot @htf.measures( htf.Measurement("firmware_version"), htf.Measurement("boot_time") .in_range(maximum=2000), htf.Measurement("self_test_pass") .equals(True), ) def boot_validation(test): test.measurements.firmware_version = "3.1.0-rc2" test.measurements.boot_time = 843 test.measurements.self_test_pass = True @htf.measures( htf.Measurement("ble_rssi") .in_range(minimum=-70, maximum=-20), htf.Measurement("wifi_throughput") .in_range(minimum=50.0), ) def wireless_check(test): test.measurements.ble_rssi = -42 test.measurements.wifi_throughput = 87.3 def main(): test = htf.Test( boot_validation, wireless_check, station_id="STATION-FT-04", ) with TofuPilot(test): test.execute(test_start=lambda: "HUB-2026-03-00891") if __name__ == "__main__": main() ``` Every run in TofuPilot will carry the firmware version as a searchable field. You can filter your run history by this value directly from the dashboard. ## Reading Firmware Version from the DUT In production, you shouldn't hardcode the firmware version. Read it from the device during the test so the record always reflects what's actually on the unit. ```python filename="firmware_read_test.py" import openhtf as htf from openhtf.plugs import BasePlug from tofupilot.openhtf import TofuPilot class DeviceConnection(BasePlug): """Connects to the DUT over serial to read device info.""" def read_firmware_version(self): # Replace with your actual device communication # e.g., send AT command, read UART, query USB descriptor return "3.1.0-rc2" def read_bootloader_version(self): return "1.2.0" def tearDown(self): pass @htf.measures( htf.Measurement("firmware_version"), htf.Measurement("bootloader_version"), ) @htf.plug(device=DeviceConnection) def read_device_info(test, device): fw = device.read_firmware_version() bl = device.read_bootloader_version() test.measurements.firmware_version = fw test.measurements.bootloader_version = bl @htf.measures( htf.Measurement("adc_accuracy_percent") .in_range(maximum=0.5), htf.Measurement("watchdog_recovery_pass") .equals(True), ) def functional_test(test): test.measurements.adc_accuracy_percent = 0.12 test.measurements.watchdog_recovery_pass = True def main(): test = htf.Test( read_device_info, functional_test, station_id="STATION-FT-01", ) with TofuPilot(test): test.execute(test_start=lambda: "SENS-2026-04217") if __name__ == "__main__": main() ``` This approach records the firmware version as a measurement (visible in the run detail) and ensures TofuPilot captures exactly what was on the device, not what you expected to be on it. ## Filtering Runs by Firmware Version Once your runs carry firmware version data, TofuPilot's dashboard lets you filter by it. This is where the tracking pays off. **Failure investigation.** Filter runs to a specific firmware version and check FPY. If v3.1.0-rc2 has a lower yield than v3.0.9, you've found your suspect. **Regulatory evidence.** During an audit, filter by firmware version to show that all units shipped with the validated release. Export the filtered view as evidence. **Field return correlation.** When a customer reports a failure, look up the unit's serial number, find the firmware version from the test record, and check if other units with the same firmware show similar behavior. ## What to Track Beyond Firmware Firmware version is the most critical, but consider tracking other software and configuration data that affects test outcomes. | Field | Example Value | Why It Matters | |---|---|---| | `firmware_version` | `3.1.0-rc2` | Core software on the DUT | | `bootloader_version` | `1.2.0` | Can affect boot behavior and update paths | | `hardware_revision` | `rev-C` | PCB changes affect test results | | `config_profile` | `US-120V` | Regional or customer-specific settings | | `test_script_version` | `2.0.4` | Proves which test logic was used | Record these as measurements in your test phases so they're stored with every run in TofuPilot. ### Anomaly Detection in Hardware Test Data URL: https://www.tofupilot.com/guides/anomaly-detection-in-hardware-test-data Learn how to detect anomalies in hardware test measurements using TofuPilot's analytics and automated limit checking. # Anomaly Detection in Hardware Test Data A unit passes all its limits but something looks wrong. The voltage is 2% higher than usual. The calibration took twice as long. These subtle anomalies are invisible in pass/fail reports but obvious in measurement trends. TofuPilot surfaces them automatically. ## Why Pass/Fail Isn't Enough Spec limits define the acceptable range. But "within spec" doesn't mean "normal." A power supply output that's been stable at 3.30V for 10,000 units and suddenly reads 3.34V is still within the 3.25V-3.35V spec, but it's anomalous. Something changed. Catching these anomalies early prevents escapes. The unit that reads 3.34V today might read 3.36V tomorrow, after it's shipped. ## Types of Hardware Test Anomalies | Anomaly Type | Example | Risk | |-------------|---------|------| | Drift | Measurement gradually shifting toward a limit | Eventual field failure | | Step change | Measurement jumping to a new baseline | Process or tooling change | | Increased variance | Measurement scatter widening | Loss of process control | | Outlier | Single unit far from the distribution | Component defect | | Bimodal distribution | Two clusters instead of one | Mixed component lots | ## Setting Up Anomaly Detection in TofuPilot ### Step 1: Define Measurements with Limits Every measurement you upload should include limits. TofuPilot uses these for pass/fail gating, but they also define the baseline for anomaly detection. ```python filename="measurements_with_limits.py" from tofupilot import TofuPilotClient client = TofuPilotClient() client.create_run( procedure_id="SENSOR-CALIBRATION", unit_under_test={"serial_number": "SENS-7821"}, steps=[{ "name": "Zero Offset", "step_type": "measurement", "status": True, "measurements": [{ "name": "zero_offset_mv", "value": 0.12, "unit": "mV", "limit_low": -1.0, "limit_high": 1.0, }], }, { "name": "Sensitivity", "step_type": "measurement", "status": True, "measurements": [{ "name": "sensitivity_mv_per_g", "value": 100.3, "unit": "mV/g", "limit_low": 95.0, "limit_high": 105.0, }], }], ) ``` ### Step 2: Monitor Measurement Distributions Open the measurement view for any procedure in TofuPilot. The histogram shows the distribution of values across all runs. A healthy process looks like a tight normal distribution centered well within the spec limits. Warning signs: - **Distribution shifting**: The center of the histogram is moving toward one limit - **Distribution widening**: The tails are getting closer to the limits - **Multiple peaks**: Two or more clusters in the histogram ### Step 3: Set Warning Thresholds Don't wait for a measurement to hit its spec limit. Set warning thresholds at a percentage of the spec range. | Threshold | Level | Action | |-----------|-------|--------| | Within 70% of spec range | Normal | No action | | 70-90% of spec range | Warning | Investigate trend | | 90-100% of spec range | Critical | Stop and root-cause | | Beyond spec range | Fail | Unit rejected | ### Step 4: Track Trends Over Time The measurement trend view in TofuPilot shows each reading plotted over time. Use this to spot: - **Slow drift**: A linear trend heading toward a limit. Often caused by fixture wear, calibration drift, or environmental changes. - **Step changes**: An abrupt shift in the baseline. Often caused by a new component lot, process change, or equipment swap. - **Periodic patterns**: Readings that cycle with time of day, day of week, or production batch. Often caused by environmental factors (temperature, humidity) or shift-dependent operator procedures. ## Automated Data Checks TofuPilot checks every incoming measurement against its defined limits in real time. Failed measurements are flagged immediately, and the run is marked as failed. For anomaly detection beyond simple limit checks, export measurement data via the API and apply statistical methods: ```python filename="cpk_analysis.py" import numpy as np # Fetch measurement values from TofuPilot API values = [3.30, 3.31, 3.29, 3.30, 3.32, 3.31, 3.30, 3.34, 3.29, 3.31] usl = 3.35 # Upper spec limit lsl = 3.25 # Lower spec limit mean = np.mean(values) std = np.std(values, ddof=1) cpk = min((usl - mean) / (3 * std), (mean - lsl) / (3 * std)) print(f"Cpk: {cpk:.2f}") if cpk < 1.33: print("Process capability below target. Investigate.") ``` ## Real-World Example: Detecting a Supplier Quality Escape A robotics company tests motor controllers on their production line. Every unit measures motor current draw at three load points. For 6 months, the 50% load measurement averaged 2.1A with a standard deviation of 0.05A. In week 27, the average shifted to 2.25A. Still within the 1.5A-3.0A spec limit, so no units failed. But TofuPilot's trend view showed the step change clearly. Investigation revealed a new motor driver IC lot with slightly different gate threshold voltages. The company worked with their supplier to tighten the incoming spec before the drift could cause field issues. Without measurement trending, this would have been invisible until units started failing in the field. ### Use pytest for Hardware Testing URL: https://www.tofupilot.com/guides/how-to-use-pytest-for-hardware-testing-with-tofupilot Learn how to use pytest to automate hardware tests and upload results to TofuPilot for centralized tracking and analytics. # How to Use pytest for Hardware Testing with TofuPilot pytest is the most popular Python testing framework. You already know it from software testing. You can use it for hardware testing too. This guide shows how to structure hardware tests with pytest and upload results to TofuPilot. ## Why pytest for Hardware | Feature | How it helps hardware testing | |---------|------------------------------| | Fixtures | Set up instruments, power supplies, connections | | Parametrize | Run the same test across multiple DUTs or configurations | | Markers | Tag tests by type (functional, safety, calibration) | | Assertions | Define pass/fail criteria for measurements | | Plugins | Extend with custom reporting, parallel execution | pytest gives you the test runner. TofuPilot gives you the data platform. Together they replace ad-hoc scripts and Excel files. ## Basic Hardware Test with pytest ```python filename="test_power_rails.py" import pytest import pyvisa from tofupilot import TofuPilotClient # Fixtures for instrument setup @pytest.fixture(scope="session") def rm(): return pyvisa.ResourceManager() @pytest.fixture(scope="session") def dmm(rm): inst = rm.open_resource("TCPIP::192.168.1.10::INSTR") yield inst inst.close() @pytest.fixture(scope="session") def psu(rm): inst = rm.open_resource("TCPIP::192.168.1.11::INSTR") inst.write("OUTP ON") yield inst inst.write("OUTP OFF") inst.close() @pytest.fixture(scope="session") def tofupilot(): return TofuPilotClient() # Tests def test_vcc_3v3(dmm, tofupilot): voltage = float(dmm.query("MEAS:VOLT:DC?")) assert 3.25 <= voltage <= 3.35, f"3.3V rail out of spec: {voltage}V" def test_vcc_1v8(dmm, tofupilot): voltage = float(dmm.query("MEAS:VOLT:DC?")) assert 1.75 <= voltage <= 1.85, f"1.8V rail out of spec: {voltage}V" def test_idle_current(dmm, tofupilot): current = float(dmm.query("MEAS:CURR:DC?")) * 1000 # mA assert 30 <= current <= 60, f"Idle current out of spec: {current}mA" ``` ## Uploading pytest Results to TofuPilot ### Option 1: Upload in a Fixture (Recommended) Collect measurements during tests and upload at the end. ```python filename="conftest.py" import pytest from tofupilot import TofuPilotClient class TestCollector: """Collects measurements during a test session.""" def __init__(self): self.measurements = [] self.steps = [] self.all_passed = True def add_measurement(self, step_name, name, value, unit, limit_low=None, limit_high=None): passed = True if limit_low is not None and value < limit_low: passed = False if limit_high is not None and value > limit_high: passed = False if not passed: self.all_passed = False # Find or create step step = next((s for s in self.steps if s["name"] == step_name), None) if not step: step = {"name": step_name, "step_type": "measurement", "status": True, "measurements": []} self.steps.append(step) measurement = {"name": name, "value": value, "unit": unit} if limit_low is not None: measurement["limit_low"] = limit_low if limit_high is not None: measurement["limit_high"] = limit_high step["measurements"].append(measurement) if not passed: step["status"] = False @pytest.fixture(scope="session") def collector(): return TestCollector() @pytest.fixture(scope="session", autouse=True) def upload_results(collector): yield # Run all tests # Upload after all tests complete client = TofuPilotClient() serial = input("Enter DUT serial: ") if not hasattr(collector, "serial") else collector.serial client.create_run( procedure_id="PYTEST-BOARD-FUNCTIONAL", unit_under_test={"serial_number": serial}, run_passed=collector.all_passed, steps=collector.steps, ) ``` ```python filename="test_board.py" def test_vcc_3v3(dmm, collector): voltage = float(dmm.query("MEAS:VOLT:DC?")) collector.add_measurement("Power Rails", "vcc_3v3", voltage, "V", limit_low=3.25, limit_high=3.35) assert 3.25 <= voltage <= 3.35 def test_vcc_1v8(dmm, collector): voltage = float(dmm.query("MEAS:VOLT:DC?")) collector.add_measurement("Power Rails", "vcc_1v8", voltage, "V", limit_low=1.75, limit_high=1.85) assert 1.75 <= voltage <= 1.85 def test_idle_current(dmm, collector): current = float(dmm.query("MEAS:CURR:DC?")) * 1000 collector.add_measurement("Current Draw", "idle_current_ma", current, "mA", limit_low=30, limit_high=60) assert 30 <= current <= 60 ``` ### Option 2: pytest Plugin Write a pytest plugin that hooks into test results. ```python filename="conftest.py" import pytest from tofupilot import TofuPilotClient def pytest_sessionfinish(session, exitstatus): """Upload results after all tests complete.""" client = TofuPilotClient() passed = exitstatus == 0 # Collect results from the session steps = [] for item in session.items: report = item.stash.get("report", None) if report: steps.append({ "name": item.name, "step_type": "measurement", "status": report.passed, "measurements": item.stash.get("measurements", []), }) client.create_run( procedure_id="PYTEST-FUNCTIONAL", unit_under_test={"serial_number": session.config.getoption("--serial", "UNKNOWN")}, run_passed=passed, steps=steps, ) ``` ## Running Tests ```bash filename="terminal" # Run all hardware tests pytest test_board.py -v # Run with serial number pytest test_board.py --serial UNIT-5501 # Run only power rail tests pytest test_board.py -k "vcc" -v # Run with markers pytest test_board.py -m "safety" -v ``` ## Organizing Hardware Tests with pytest ### Use Markers for Test Categories ```python filename="test_board.py" import pytest @pytest.mark.safety def test_hipot(hipot_tester, collector): """Safety test - must pass for every unit.""" leakage = hipot_tester.run_test(1500) collector.add_measurement("Safety", "hipot_leakage_ma", leakage, "mA", limit_high=5.0) assert leakage < 5.0 @pytest.mark.functional def test_communication(dut, collector): """Functional test - verifies basic operation.""" response = dut.ping() collector.add_measurement("Communication", "uart_ping", 1 if response else 0, "bool", limit_low=1) assert response @pytest.mark.calibration def test_adc_accuracy(dut, collector): """Calibration test - verifies measurement accuracy.""" error = dut.measure_adc_error() collector.add_measurement("Calibration", "adc_error_pct", error, "%", limit_high=0.5) assert error < 0.5 ``` ### Use Parametrize for Multi-Channel Tests ```python filename="test_multi_channel.py" import pytest @pytest.mark.parametrize("channel,expected_v,tolerance", [ (1, 3.3, 0.05), (2, 1.8, 0.05), (3, 5.0, 0.10), (4, 12.0, 0.20), ]) def test_voltage_rail(dmm, collector, channel, expected_v, tolerance): voltage = dmm.measure_channel(channel) collector.add_measurement( "Power Rails", f"rail_ch{channel}_v", voltage, "V", limit_low=expected_v - tolerance, limit_high=expected_v + tolerance, ) assert abs(voltage - expected_v) < tolerance ``` ## pytest vs. OpenHTF | Feature | pytest | OpenHTF | |---------|--------|---------| | Learning curve | Low (most Python devs know it) | Medium (hardware-specific) | | Hardware test features | General-purpose | Purpose-built (phases, plugs, measurements) | | Operator UI | None built-in | Built-in web UI | | Community | Massive | Small but focused | | TofuPilot integration | Via custom code | Native callback | Use pytest when your team already knows it and you want something running quickly. Use OpenHTF when you need the full hardware test framework with operator interface and structured phases. Both upload to TofuPilot the same way. The data in TofuPilot looks identical regardless of which framework generated it. ### What Is Ongoing Reliability Testing (ORT) URL: https://www.tofupilot.com/guides/what-is-ort-with-tofupilot Ongoing reliability testing (ORT) samples production units to catch reliability drift. Learn how ORT works and how to track it with TofuPilot. # What Is ORT with TofuPilot Ongoing reliability testing (ORT) pulls units from the production line at regular intervals and subjects them to stress tests. It catches reliability problems caused by process drift, supplier changes, or material variation that production tests don't detect. This guide covers how ORT works, what it catches, and how to track ORT results with TofuPilot. ## Why Production Test Is Not Enough Production tests verify that a unit works right now. They don't tell you whether it will still work in a year. A solder joint can pass ICT and FCT but crack after 200 thermal cycles. A capacitor can measure within spec but degrade under humidity. ORT catches these problems by: 1. Sampling units from each production lot 2. Running stress tests (temperature, humidity, vibration, powered operation) 3. Comparing results to the reliability baseline established during DVT If ORT results drift from baseline, something changed in the process or supply chain. ## What ORT Catches | Problem | How It Shows Up in ORT | |---------|----------------------| | Solder process drift | Increased failures during thermal cycling | | Component substitution | Different failure mode than baseline | | Contamination | Humidity test failures, leakage current increase | | Fixture wear | Marginal contact resistance readings | | Supplier quality change | Shifted measurement distributions | ## Typical ORT Program | Parameter | Typical Value | |-----------|--------------| | Sample size | 2-5 units per lot or per week | | Stress profile | Subset of HALT/DVT stress levels | | Duration | 48-168 hours per sample | | Functional checks | Before, during (at intervals), and after stress | | Acceptance criteria | Zero failures, measurements within DVT baseline | ## Prerequisites - Python 3.10+ - OpenHTF installed (`pip install openhtf`) - TofuPilot Python SDK installed (`pip install tofupilot`) ## Step 1: Define ORT Functional Checks ORT functional checks run before stress, at intervals during stress, and after stress. They should match the measurements used in your DVT baseline. ```python filename="ort_check.py" import openhtf as htf from openhtf.util import units @htf.measures( htf.Measurement("output_voltage_V") .in_range( minimum=4.85, maximum=5.15, marginal_minimum=4.9, marginal_maximum=5.1, ) .with_units(units.VOLT), htf.Measurement("quiescent_current_uA") .in_range(maximum=50) .with_units(units.MICROAMPERE), ) def phase_electrical(test): """Core electrical parameters for ORT baseline comparison.""" test.measurements.output_voltage_V = 5.01 test.measurements.quiescent_current_uA = 12.4 @htf.measures( htf.Measurement("boot_time_ms") .in_range(maximum=1500) .with_units(units.MILLISECOND), htf.Measurement("memory_check").equals("PASS"), ) def phase_functional(test): """Functional checks to detect degradation after stress.""" test.measurements.boot_time_ms = 980 test.measurements.memory_check = "PASS" ``` ## Step 2: Log Each Check to TofuPilot Run the ORT check script at each interval. Each execution creates a new run linked to the unit's serial number. ```python filename="ort_check.py" from tofupilot.openhtf import TofuPilot test = htf.Test( phase_electrical, phase_functional, ) with TofuPilot(test): test.execute(test_start=lambda: input("Scan ORT sample serial: ")) ``` ## Step 3: Monitor ORT Trends in TofuPilot TofuPilot tracks ORT results alongside production test data. Open the Analytics tab to see: - **Measurement trends** per ORT sample over time - **Baseline comparison** between ORT results and DVT data - **Marginal results** flagged before they become failures - **Control charts** showing process stability across production lots When an ORT sample shows measurements drifting toward limits, investigate before the next production lot ships. The earlier you catch process drift, the smaller the containment scope. ## ORT vs Other Reliability Methods | Method | When | Units | Goal | |--------|------|-------|------| | HALT | Design (EVT/DVT) | 5-15 prototypes | Find design limits | | ALT | Pre-production (DVT) | 20-50 units | Predict field life | | HASS | Production | Every unit | Screen latent defects | | ORT | Production (ongoing) | 2-5 per lot | Monitor reliability drift | | Burn-in | Production | Every unit or sample | Screen infant mortality | ORT is the ongoing check that validates your HALT margins and HASS screens are still working. If ORT starts failing, your HASS profile may need updating, or your process needs investigation. ### Migrate from Custom Test Databases URL: https://www.tofupilot.com/guides/how-to-migrate-from-custom-test-databases-to-tofupilot Replace your custom PostgreSQL, SQLite, or Access test database with structured test data collection using OpenHTF and TofuPilot. Many hardware teams build custom databases for test results. PostgreSQL, SQLite, Microsoft Access. They work well at first. Then someone asks for FPY trends across stations, or you need multi-site access, or an auditor wants full revision history, and the maintenance burden outweighs the original simplicity. TofuPilot replaces that custom infrastructure with a managed platform that handles storage, analytics, and traceability out of the box. ## Common Custom DB Patterns Most homegrown test databases follow one of a few schemas. Here's a typical one: ```sql filename="typical_test_schema.sql" -- The pattern most teams converge on CREATE TABLE test_runs ( id SERIAL PRIMARY KEY, serial_number VARCHAR(50), test_name VARCHAR(100), result VARCHAR(10), -- 'PASS' or 'FAIL' operator VARCHAR(50), station VARCHAR(50), started_at TIMESTAMP, duration_seconds FLOAT ); CREATE TABLE measurements ( id SERIAL PRIMARY KEY, run_id INTEGER REFERENCES test_runs(id), name VARCHAR(100), value FLOAT, unit VARCHAR(20), lower_limit FLOAT, upper_limit FLOAT, passed BOOLEAN ); ``` This structure captures the basics. But it misses things that matter at scale: - **No built-in analytics.** FPY, Cpk, and control charts require custom queries and visualization code that someone has to maintain. - **No audit trail.** If someone updates a row, the original value is gone unless you've built trigger-based history tables. - **No multi-site access.** A local PostgreSQL or SQLite file doesn't serve teams across locations without infrastructure work. - **Schema drift.** Each test station might insert slightly different data depending on who wrote the INSERT statement. ## TestStand Database Users If you're using NI TestStand's built-in database logger, you have a variation of this problem. TestStand writes to SQL Server, Oracle, or Access using a fixed schema with `UUT_RESULT`, `STEP_RESULT`, `PROP_RESULT`, and type-specific tables (`PROP_NUMERICLIMIT`, `PROP_NUMERIC`, `PROP_STRINGVALUE`). Querying it requires 5-6 table JOINs, and adding custom metadata means modifying the Process Model's database mapping. The same migration pattern applies: replace the database INSERT (or TestStand's database logger) with OpenHTF measurements and let TofuPilot handle storage. If you're migrating from TestStand specifically, see the dedicated guide on migrating from NI TestStand to Python. ## Replacing Custom Inserts with OpenHTF If your current workflow looks like "run test script, INSERT results into database," here's the OpenHTF + TofuPilot equivalent. Instead of managing the database, you define measurements as part of the test and let TofuPilot handle storage. A typical custom database insert might look like this: ```python filename="old_custom_insert.py" # What you're replacing: manual DB inserts after each test import psycopg2 conn = psycopg2.connect("dbname=testdata user=testeng") cur = conn.cursor() cur.execute( "INSERT INTO test_runs (serial_number, test_name, result, station) " "VALUES (%s, %s, %s, %s) RETURNING id", ("SN-5001", "thermal_cycle", "PASS", "STATION-3"), ) run_id = cur.fetchone()[0] cur.execute( "INSERT INTO measurements (run_id, name, value, unit, lower_limit, upper_limit, passed) " "VALUES (%s, %s, %s, %s, %s, %s, %s)", (run_id, "peak_temp", 84.2, "C", 70.0, 90.0, True), ) conn.commit() conn.close() ``` Here's the same test in OpenHTF with TofuPilot: ```python filename="thermal_cycle_test.py" import openhtf as htf from openhtf.util import units from tofupilot.openhtf import TofuPilot @htf.measures( htf.Measurement("peak_temp") .with_units(units.DEGREE_CELSIUS) .in_range(70.0, 90.0), htf.Measurement("settling_time") .with_units(units.SECOND) .in_range(maximum=30.0), htf.Measurement("thermal_resistance") .in_range(0.5, 2.0), ) def thermal_cycle(test): test.measurements.peak_temp = 84.2 test.measurements.settling_time = 18.7 test.measurements.thermal_resistance = 1.1 def main(): test = htf.Test(thermal_cycle) with TofuPilot(test): test.execute(test_start=lambda: "SN-5001") if __name__ == "__main__": main() ``` No database connection code. No SQL. No schema maintenance. Measurements are defined with their limits in the test itself, so every run is validated and structured identically. ## Migrating Historical Data You have years of test data in your custom database. TofuPilot's REST API lets you import it with full timestamp preservation. ```python filename="import_from_custom_db.py" import psycopg2 from tofupilot import TofuPilotClient client = TofuPilotClient() conn = psycopg2.connect("dbname=testdata user=testeng") cur = conn.cursor() # Fetch runs with their measurements cur.execute(""" SELECT r.serial_number, r.test_name, r.result, r.started_at, r.duration_seconds, m.name, m.value, m.unit, m.lower_limit, m.upper_limit FROM test_runs r JOIN measurements m ON m.run_id = r.id ORDER BY r.id """) current_run = None steps = [] for row in cur: serial, test_name, result, started_at, duration, m_name, m_val, m_unit, m_low, m_high = row if current_run and current_run != (serial, started_at): # Flush previous run prev_serial, prev_started = current_run client.create_run( procedure_id=test_name, unit_under_test={"serial_number": prev_serial}, run_passed=result == "PASS", started_at=prev_started, duration=duration, steps=steps, ) steps = [] current_run = (serial, started_at) # Find or create step step = next((s for s in steps if s["name"] == test_name), None) if not step: step = {"name": test_name, "step_passed": result == "PASS", "measurements": []} steps.append(step) step["measurements"].append({ "name": m_name, "measured_value": m_val, "unit": m_unit, "lower_limit": m_low, "upper_limit": m_high, }) conn.close() ``` Adapt the query to match your schema. The key is mapping your tables to TofuPilot's structure: each row in `test_runs` becomes a run, each row in `measurements` becomes a measurement within a step. ## What Replaces Your Custom Queries If you've built custom SQL queries or Python scripts for analytics, TofuPilot's dashboard replaces them. | Your custom query / script | TofuPilot built-in feature | |---|---| | `SELECT COUNT(*) ... GROUP BY result` for yield | FPY trends, updated in real time | | Statistical process control scripts | Cpk and control charts per measurement | | `WHERE result = 'FAIL' GROUP BY measurement` | Failure Pareto with drill-down | | `GROUP BY station` for throughput | Station throughput dashboard | | `WHERE serial_number = 'X'` for traceability | Serial number search with full history | | Custom reporting scripts | Exportable reports and REST API | TofuPilot tracks all of this automatically. Open the Analytics tab to see FPY, Cpk, and failure analysis for any procedure. ## Running Both Systems During Transition You can run your custom database and TofuPilot in parallel to validate the migration: 1. **Add TofuPilot to one test station** while keeping your existing database inserts active. Both systems receive the same results. 2. **Compare data** for a week. Check that measurement values, pass/fail counts, and timestamps match between your database and TofuPilot. 3. **Remove the custom database inserts** from that station once you're confident. Move to the next station. 4. **Keep your old database read-only** as an archive. You can always query it for historical verification. The custom database doesn't need to go away immediately. It just stops being the system of record once TofuPilot takes over. ### RF Testing and Calibration with TofuPilot URL: https://www.tofupilot.com/guides/rf-testing-and-calibration-with-tofupilot Learn how to log RF test and calibration data for wireless products using TofuPilot, covering output power, sensitivity, and frequency accuracy. # RF Testing and Calibration with TofuPilot Every wireless product needs RF testing: output power, receiver sensitivity, frequency accuracy, spurious emissions. For products with WiFi, Bluetooth, LoRa, cellular, or custom RF, these measurements determine whether the product meets regulatory requirements and actually works in the field. TofuPilot stores RF test data for trending, calibration tracking, and compliance. ## RF Test Parameters | Parameter | What it measures | Why it matters | |-----------|-----------------|---------------| | TX output power (dBm) | Transmitted signal strength | Too high: regulatory violation. Too low: poor range | | RX sensitivity (dBm) | Minimum receivable signal | Determines receive range | | Frequency error (ppm) | Crystal/oscillator accuracy | Affects interoperability | | EVM (Error Vector Magnitude) | Modulation quality | Affects data throughput | | Spurious emissions | Unintended RF output | Regulatory compliance | | RSSI accuracy | Received signal strength indicator | Affects link management | ## Logging RF Test Results ### Basic RF Parametric Test ```python filename="rf_test.py" from tofupilot import TofuPilotClient client = TofuPilotClient() def rf_production_test(serial, tx_power, rx_sensitivity, freq_error_ppm, evm_pct): client.create_run( procedure_id="RF-PARAMETRIC-BLE", unit_under_test={ "serial_number": serial, "part_number": "IOT-SENSOR-V3", }, run_passed=True, steps=[{ "name": "TX Performance", "step_type": "measurement", "status": 0 <= tx_power <= 4, "measurements": [ {"name": "tx_power_dbm", "value": tx_power, "unit": "dBm", "limit_low": 0, "limit_high": 4}, {"name": "freq_error_ppm", "value": freq_error_ppm, "unit": "ppm", "limit_low": -20, "limit_high": 20}, {"name": "evm_pct", "value": evm_pct, "unit": "%", "limit_high": 30}, ], }, { "name": "RX Performance", "step_type": "measurement", "status": rx_sensitivity <= -90, "measurements": [ {"name": "rx_sensitivity_dbm", "value": rx_sensitivity, "unit": "dBm", "limit_high": -90}, ], }], ) ``` ### Multi-Channel RF Test Test across all operating channels to catch channel-specific issues. ```python filename="rf_multi_channel.py" # BLE has 40 channels (0-39) # Test a representative set test_channels = [0, 12, 19, 20, 38, 39] measurements = [] for ch in test_channels: set_channel(ch) power = measure_tx_power() measurements.append({ "name": f"tx_power_ch{ch}_dbm", "value": power, "unit": "dBm", "limit_low": 0, "limit_high": 4, }) client.create_run( procedure_id="RF-MULTI-CHANNEL-BLE", unit_under_test={"serial_number": serial}, run_passed=all(0 <= m["value"] <= 4 for m in measurements), steps=[{ "name": "Multi-Channel TX Power", "step_type": "measurement", "status": True, "measurements": measurements, }], ) ``` ## RF Calibration Many wireless products require per-unit RF calibration to compensate for component tolerances. The calibration process: 1. Measure actual TX power at a reference level 2. Calculate the offset from target 3. Write a calibration value to the DUT's flash/EEPROM 4. Verify calibrated output matches target ```python filename="rf_calibration.py" def calibrate_rf(serial): # Step 1: Measure uncalibrated power raw_power = measure_tx_power() # Step 2: Calculate calibration offset target_power = 0.0 # dBm cal_offset = target_power - raw_power # Step 3: Write calibration to DUT write_cal_value(cal_offset) # Step 4: Verify calibrated_power = measure_tx_power() client.create_run( procedure_id="RF-CALIBRATION-BLE", unit_under_test={"serial_number": serial}, run_passed=abs(calibrated_power - target_power) < 1.0, steps=[{ "name": "TX Calibration", "step_type": "measurement", "status": True, "measurements": [ {"name": "raw_tx_power_dbm", "value": raw_power, "unit": "dBm"}, {"name": "cal_offset_db", "value": cal_offset, "unit": "dB"}, {"name": "calibrated_tx_power_dbm", "value": calibrated_power, "unit": "dBm", "limit_low": -1.0, "limit_high": 1.0}, ], }], ) ``` ## Tracking RF Performance Across Production ### Calibration Offset Distribution The distribution of calibration offsets tells you about your RF hardware consistency. A tight distribution (small offsets) means consistent PCB manufacturing. A wide or drifting distribution suggests: - Antenna placement variation - PCB impedance variation - Component lot changes (balun, matching network) ### TX Power Over Time Plot TX power across production. Look for: | Pattern | Cause | |---------|-------| | Gradual drift | Test equipment calibration drift | | Step change | New component lot, PCB revision | | Increased variance | Manufacturing process variation | | Channel-dependent failures | Antenna or matching network issues | ### RX Sensitivity Monitoring Receiver sensitivity is harder to measure in production (requires a shielded environment and calibrated signal source). Track it to catch: - LNA gain degradation - Filter mistuning - Increased noise figure from layout changes ## Test Fixture Considerations RF testing requires controlled RF environments: | Setup | Use case | Isolation | |-------|----------|-----------| | Shielded box | Production testing | 60-80 dB | | Anechoic chamber | Antenna characterization | 80+ dB | | RF cable (conducted) | Board-level testing | Direct connection | Log the fixture type and serial number with each test run. If RF test results shift, the fixture calibration or cable condition may have changed. ## Regulatory Compliance Data RF certification (FCC, CE, IC, MIC) requires specific test data. While certification testing is done at accredited labs, pre-compliance data from TofuPilot helps you: - Predict certification outcomes based on production margins - Compare production units to the original certified unit - Track any changes that could void certification ### OpenHTF vs NI TestStand for Manufacturing Test URL: https://www.tofupilot.com/guides/openhtf-vs-ni-teststand-with-tofupilot Compare OpenHTF and NI TestStand for manufacturing test automation, with feature matrices, code examples, cost analysis, and database integration differences. OpenHTF is a free, open-source test framework from Google. NI TestStand is a commercial test sequencer from NI (Emerson) at $4,310/seat/year. Both run manufacturing tests with measurements, limits, and sequencing. They differ in cost, platform support, database integration, and how you manage test data. This guide compares them side by side with real code and concrete tradeoffs. ## Feature Comparison | Feature | OpenHTF | NI TestStand | |---------|---------|-------------| | Language | Python | LabVIEW, C, .NET, Python | | License | Apache 2.0 (free) | $4,310/seat/year | | Platform | Linux, macOS, Windows | Windows only | | Test editor | Any code editor | Proprietary Sequence Editor | | Version control | Git (plain .py files) | Difficult (binary .seq files) | | Structured measurements | Built-in (name, value, limits, units) | Built-in (Numeric Limit, String Value) | | Serial number input | Built-in prompt | Built-in (Process Model) | | Parallel DUT | Limited | Native | | Instrument drivers | PyVISA, pyserial, nidaqmx | NI VISA, IVI, NI drivers | | Database logging | TofuPilot (1 line) | Built-in (complex schema) | | Analytics (FPY, Cpk) | TofuPilot (automatic) | Custom queries or third-party | | CI/CD integration | Native (Python) | Limited (added 2025 Q2) | | Report generation | TofuPilot (automatic) | Built-in XML/HTML | | Community | Small (~640 GitHub stars) | Large (NI forums, training courses) | | Learning curve | Medium | High | ## Cost Analysis | Metric | OpenHTF + TofuPilot | NI TestStand | |--------|-------------------|-------------| | 5 seats, 1 year | $0 (TofuPilot Lab is free) | $21,550 | | 20 seats, 1 year | $0 | $86,200 | | Runtime deployment | Free | Additional runtime licenses | | Training | Self-taught (openhtf.com docs) | NI courses ($2,000+) | | Vendor lock-in | None | High (NI ecosystem) | TestStand also requires Windows, which means Windows licenses for every test station. OpenHTF runs on Linux, which is free and more stable for long-running production stations. ## The Same Test in Both Frameworks A simple functional test: measure a 3.3V rail, check it's within 3.2V to 3.4V. ### OpenHTF + TofuPilot ```python filename="openhtf_power_test.py" import openhtf as htf from openhtf.util import units from tofupilot.openhtf import TofuPilot @htf.measures( htf.Measurement("rail_3v3") .in_range(3.2, 3.4) .with_units(units.VOLT), ) def test_power_rail(test): voltage = 5.02 # Replace with instrument read test.measurements.rail_3v3 = voltage def main(): test = htf.Test( test_power_rail, procedure_id="FCT-001", part_number="PCBA-100", ) with TofuPilot(test): test.execute(test_start=lambda: input("Scan serial: ")) if __name__ == "__main__": main() ``` Measurements are structured data: name, value, limits, units. TofuPilot integration is one `with` statement. The operator gets a serial number prompt automatically. ### NI TestStand In TestStand, you create a sequence file (.seq) in the Sequence Editor. Add a "Numeric Limit Test" step, set the test expression to your instrument read, configure Low Limit = 3.2 and High Limit = 3.4. Connect the step to a Code Module (LabVIEW VI, DLL, or .NET assembly) that talks to the instrument. The test logic lives in the Code Module. The sequencing, limits, and reporting live in the .seq file. You can't see both in a single text file, and you can't diff the .seq file in Git. ## Concept Mapping Every TestStand concept has a direct OpenHTF equivalent. | TestStand | OpenHTF | |-----------|---------| | Sequence file (.seq) | Python test script (.py) | | Sequence Editor (GUI) | Any code editor (VS Code, PyCharm) | | Step | Phase function | | Numeric Limit Test | `htf.Measurement("name").in_range(low, high)` | | String Value Test | `htf.Measurement("name").equals("expected")` | | Pass/Fail Test | `htf.Measurement("name").equals(True)` | | Code Module (DLL, VI) | Plug class | | FileGlobals / StationGlobals | Plug instance attributes | | Process Model | TofuPilot integration | | Setup / Cleanup groups | First/last phase functions, or PhaseGroups | | UUT Serial Number | `test.execute(test_start=lambda: input("Scan: "))` | ## Instrument Drivers TestStand uses NI VISA and IVI drivers. OpenHTF uses PyVISA, which talks to the same instruments through the same VISA layer. If your instrument works with TestStand, it works with PyVISA. | TestStand Driver | Python Equivalent | |-----------------|-------------------| | NI VISA / IVI | PyVISA + NI-VISA backend | | NI DAQmx | nidaqmx (official NI Python package) | | NI Switch | niswitch (official NI Python package) | | NI DMM | nidmm (official NI Python package) | | Serial / UART | pyserial | | Custom DLL | ctypes or cffi | NI publishes official Python packages for most of their hardware. You don't lose instrument support by switching to Python. ### OpenHTF Plug for an Instrument TestStand wraps instruments in Code Modules. OpenHTF wraps them in Plugs, which have automatic lifecycle management (setUp/tearDown). ```python filename="plugs/multimeter.py" import pyvisa from openhtf.plugs import BasePlug import openhtf as htf from openhtf.util import units from tofupilot.openhtf import TofuPilot class MultimeterPlug(BasePlug): """Wraps a SCPI multimeter connection.""" def setUp(self): rm = pyvisa.ResourceManager() self.instr = rm.open_resource("TCPIP::192.168.1.100::INSTR") self.instr.timeout = 5000 def measure_voltage(self, channel=1): self.instr.write(f":CONF:VOLT:DC AUTO,(@{channel})") self.instr.write(":INIT") return float(self.instr.query(":FETCH?")) def tearDown(self): self.instr.close() @htf.measures( htf.Measurement("rail_3v3") .in_range(3.2, 3.4) .with_units(units.VOLT), ) @htf.plug(dmm=MultimeterPlug) def measure_power_rail(test, dmm): test.measurements.rail_3v3 = dmm.measure_voltage(channel=1) def main(): test = htf.Test(measure_power_rail, procedure_id="FCT-001", part_number="PCBA-100") with TofuPilot(test): test.execute(test_start=lambda: input("Scan serial: ")) if __name__ == "__main__": main() ``` ## Sharing Data Between Steps TestStand uses FileGlobals and StationGlobals to pass data between steps. OpenHTF uses plug instance attributes. The plug persists for the entire test execution. ```python filename="plugs/shared_state.py" from openhtf.plugs import BasePlug import openhtf as htf from openhtf.util import units class SharedState(BasePlug): """Replaces TestStand FileGlobals and StationGlobals.""" def setUp(self): self.cal_offset = 0.0 self.firmware_version = "" def tearDown(self): pass @htf.plug(state=SharedState) def calibrate(test, state): state.cal_offset = 0.023 @htf.measures( htf.Measurement("corrected_voltage") .in_range(3.2, 3.4) .with_units(units.VOLT), ) @htf.plug(state=SharedState) def measure_corrected(test, state): raw = 3.323 # Replace with instrument read test.measurements.corrected_voltage = raw - state.cal_offset ``` ## Database and Test Data This is where the two frameworks differ most. ### TestStand Database Logging TestStand's built-in database logger writes results to SQL Server, Oracle, or Access using a fixed schema. The core tables are `UUT_RESULT`, `STEP_RESULT`, and `PROP_RESULT`, with additional tables for each property type (`PROP_NUMERICLIMIT`, `PROP_STRINGVALUE`, etc.). Querying this schema requires multi-level JOINs: | Table | Contains | Joins To | |-------|----------|----------| | `UUT_RESULT` | Serial number, overall pass/fail, timestamps | Top-level | | `STEP_RESULT` | Step name, step status | `UUT_RESULT.ID` | | `PROP_RESULT` | Property metadata | `STEP_RESULT.ID` | | `PROP_NUMERICLIMIT` | Numeric limits (low, high) | `PROP_RESULT.ID` | | `PROP_NUMERIC` | Measured numeric value | `PROP_RESULT.ID` | | `PROP_STRINGVALUE` | String comparison results | `PROP_RESULT.ID` | A simple query to get one unit's measurements requires 5-6 table JOINs. Adding custom metadata means modifying the Process Model's database schema mapping, which is fragile and hard to maintain across TestStand versions. TestStand doesn't include analytics. FPY, Cpk, control charts, and failure Pareto all require custom SQL queries or a third-party tool like WATS. ### OpenHTF + TofuPilot OpenHTF doesn't have built-in database logging. TofuPilot handles it. You add one `with TofuPilot(test):` line and every run is stored with structured measurements, limits, units, serial numbers, and metadata. | TestStand Database | TofuPilot | |---|---| | 6-table JOIN for one unit's results | Search by serial number, get full history | | Custom SQL for FPY | FPY trends, updated in real time | | No Cpk without custom code | Cpk per measurement, automatic | | No control charts | Control charts with UCL/LCL | | No failure Pareto | Failure Pareto with drill-down | | Schema locked to NI's design | Structured data, REST API access | | SQL Server/Oracle/Access | Cloud or self-hosted | | One site at a time (unless you build replication) | Multi-site from day one | No database to provision, no schema to maintain, no SQL to write. TofuPilot tracks everything automatically. Open the Analytics tab to see FPY, Cpk, and failure analysis for any procedure. ## Version Control and Git TestStand sequence files (.seq) are binary. You can store them in Git, but you can't diff them, review changes in a pull request, or merge branches. NI added a Git pane in TestStand 2025 Q2. It lets you commit, pull, and push .seq files from within the Sequence Editor. But it's file-level tracking, not content-level diffing. You can see that a file changed, not what changed inside it. NI provides a separate "Diff and Merge Utility" that can compare two .seq files visually, but it doesn't integrate with pull request workflows. OpenHTF tests are plain Python files. Standard Git workflows apply. | Capability | TestStand | OpenHTF | |---|---|---| | Commit files | Yes (Git pane, 2025 Q2) | Yes (any Git client) | | Diff changes | File-level only (Diff Utility for visual compare) | Line-by-line (`git diff`) | | Pull request review | Not possible (binary) | Standard code review | | Branch per feature | Files track, content doesn't merge | Standard | | CI/CD linting | Not possible | flake8, mypy, ruff | | Automated test of tests | Difficult | pytest on test logic | | Blame history | No | `git blame` | ### What a Python Test Diff Looks Like When you change a measurement limit in an OpenHTF test, the pull request shows exactly what changed: ```diff filename="test_power_rail.py" @htf.measures( htf.Measurement("rail_3v3") - .in_range(3.2, 3.4) + .in_range(3.1, 3.5) .with_units(units.VOLT), ) ``` Reviewers see the old limit, the new limit, and the context. In TestStand, the same change is invisible inside a binary .seq file. ## CI/CD Integration NI updated TestStand's license agreement in 2025 Q2 to allow CI/CD usage without extra cost (if you have at least one active development license). But running TestStand in CI still requires a Windows runner with TestStand installed, and the sequence files can't be linted or statically analyzed. OpenHTF tests are Python. They run in any CI system with a Python environment. | CI/CD Capability | TestStand | OpenHTF | |---|---|---| | Run in CI | Yes (2025 Q2, Windows runner required) | Yes (any OS, any CI) | | License for CI | Included with 1+ dev license (2025 Q2) | Free | | Static analysis | Not possible (.seq binary) | flake8, mypy, ruff, pylint | | Unit test on test logic | Difficult | pytest | | Lint measurement names | Not possible | Custom rules | | Docker runner | No (Windows required) | Yes | ```yaml filename=".github/workflows/test-lint.yml" # Lint and type-check OpenHTF test scripts in CI name: Test Script CI on: [push, pull_request] jobs: lint: runs-on: ubuntu-latest steps: - uses: actions/checkout@v4 - uses: actions/setup-python@v5 with: python-version: "3.12" - run: pip install ruff mypy openhtf - run: ruff check tests/ - run: mypy tests/ ``` You can catch measurement naming issues, import errors, and type problems before test scripts reach the production floor. ## When to Use Each | Scenario | Better Choice | |----------|--------------| | New project, Python team | OpenHTF + TofuPilot | | Existing NI hardware investment, large enterprise | TestStand (if it works, keep it) | | Multi-OS test stations (Linux, macOS) | OpenHTF (TestStand is Windows only) | | Budget-conscious (startup, small team) | OpenHTF + TofuPilot ($0) | | Need FPY, Cpk, analytics out of the box | OpenHTF + TofuPilot | | Deep NI PXI/CompactRIO integration | TestStand (tighter NI ecosystem) | | CI/CD and Git workflows matter | OpenHTF | | Non-NI instruments (Keysight, Rigol, R&S) | Either (both use VISA) | ## Migration Path If you're on TestStand and considering a move, the typical migration takes 4-8 weeks per test procedure. Most teams run both systems in parallel during the transition. 1. Set up Python + OpenHTF alongside TestStand on one station. 2. Convert one test procedure (the simplest one). Run both versions on the same DUTs. 3. Validate measurements match between systems. 4. Move to the next procedure once results are confirmed. 5. Decommission TestStand when all procedures are converted. The biggest risk is rushing. Convert one procedure at a time. Parallel operation is your safety net. ### The Hidden Cost of Retesting in Manufacturing URL: https://www.tofupilot.com/guides/the-hidden-cost-of-retesting-how-to-measure-it-with-tofupilot Learn how retesting inflates costs in manufacturing, identify root causes, and use TofuPilot's unit history to track and reduce retest rates. Every retest burns station time, operator attention, and margin. Most teams don't track it because retesting feels like part of the process. It's not. It's rework that compounds silently. This guide breaks down what retesting actually costs, where it comes from, and how to use TofuPilot's built-in unit history to track and reduce it. ## What Retesting Actually Costs A single retest doesn't look expensive. Multiply it by thousands of units per month and the numbers get uncomfortable. | Cost category | Per-retest estimate | At 10% retest rate (10k units/mo) | |---|---|---| | Station time (test duration) | $0.50-$2.00 | $500-$2,000/mo | | Operator handling | $0.30-$1.00 | $300-$1,000/mo | | Failure analysis (if triggered) | $5-$50 | $5,000-$50,000/mo | | Delayed shipment (opportunity cost) | Varies | Often the largest cost | | Station capacity lost | 1 slot per retest | 1,000 slots/mo unavailable | The direct cost of retesting is easy to underestimate. The indirect cost (capacity you can't use for new units) is usually worse. A test station running at 10% retest rate effectively loses 10% of its throughput. ## Common Causes of Excessive Retesting Retesting isn't always a product quality problem. Often it's a test system problem. | Root cause | Symptom | Fix | |---|---|---| | Flaky test infrastructure | Same unit passes on retry with no rework | Stabilize fixtures, tighten connections, add settling time | | Overly tight limits | Marginal units fail, pass on retest | Widen limits using production Cpk data | | Environmental sensitivity | Failures cluster at shift start or temp changes | Add environmental conditioning or guard bands | | Operator error | Failures correlate with specific operators | Improve fixturing, reduce manual steps | | Intermittent DUT defect | Unit fails randomly across multiple retests | Root cause at board level, check solder joints | | Test software bugs | Specific phases fail inconsistently | Review phase timeout, instrument communication | If your retest rate is above 5%, start with the test system before blaming the product. ## Setting Up Tests for Retest Tracking TofuPilot tracks retests by matching runs to the same unit serial number and procedure. For this to work, your test script needs to identify units correctly. ```python filename="tests/board_fct.py" import openhtf as htf from tofupilot.openhtf import TofuPilot def main(): test = htf.Test( check_power_rails, check_communication, measure_current_draw, station_id="FCT-STATION-01", ) with TofuPilot(test): test.execute(test_start=lambda: "SN-2024-00421") if __name__ == "__main__": main() ``` Two things matter for retest tracking: - **`procedure_id`** identifies the test procedure. Use a stable name that doesn't change between retests. - **`dut_id`** (returned by `test_start`) identifies the physical unit. Use the actual serial number, not a generated ID. When the same `dut_id` appears multiple times for the same `procedure_id`, TofuPilot knows it's a retest. ## Tracking Retests in TofuPilot TofuPilot tracks every run per unit serial number. You don't need to build analytics scripts or query the API to compute retest rates. **Unit history page.** Open any unit's history page to see all test attempts, ordered chronologically. A unit that was tested three times before passing shows all three runs with their outcomes. This tells you immediately whether a unit needed retesting and how many attempts it took. **Analytics tab.** Use the Analytics tab to monitor first-pass yield (FPY) trends. FPY inversely correlates with retest rate: if your FPY is 92%, roughly 8% of units needed at least one retest. Tracking FPY over time shows whether your retest problem is improving or getting worse. **Filtering by procedure and station.** You can filter analytics by procedure and station to isolate retest patterns. If one station has noticeably lower FPY than others running the same procedure, you've found a test infrastructure problem, not a product problem. ## Retest Rate vs Cost Impact Here's how retest rate maps to real production impact at different scales. | Monthly volume | Retest rate | Retests/mo | Est. cost/mo (@ $2/retest) | Station capacity lost | |---|---|---|---|---| | 1,000 | 2% | 20 | $40 | Negligible | | 1,000 | 10% | 100 | $200 | ~2 hours | | 10,000 | 2% | 200 | $400 | ~1 day | | 10,000 | 10% | 1,000 | $2,000 | ~5 days | | 100,000 | 5% | 5,000 | $10,000 | ~25 days | | 100,000 | 10% | 10,000 | $20,000 | ~50 days | At high volume, even a small retest rate improvement frees up meaningful station capacity. Going from 10% to 5% at 100k units/month recovers 25 days of station time. ## How to Reduce Retest Rate Reducing retest rate is almost always higher ROI than buying more stations. **1. Stabilize the test environment.** Most retests at low maturity come from the test system, not the product. Check fixture contact resistance, instrument warm-up, cable integrity, and software timeouts. If a unit passes on retry without rework, the problem is your test, not the DUT. **2. Use Cpk to set limits.** Limits derived from datasheets are often tighter than necessary. Pull measurement distributions from TofuPilot's analytics, calculate Cpk, and widen limits where you have margin. A Cpk of 1.33 or higher means the process is well within spec. **3. Add marginal bands.** Configure marginal limits in your test to flag units that pass but are close to the boundary. These are your future retests. Catching them early lets you investigate before the limit becomes a yield problem. **4. Root-cause the top failures.** Use TofuPilot's measurement analytics to identify which phases fail most often. Focus on the top 3. Pareto analysis almost always reveals that a small number of failure modes drive most retests. **5. Track retest rate as a KPI.** Make FPY (or its inverse, retest rate) a weekly metric. Review it in TofuPilot's Analytics tab. Teams that track retest rate consistently reduce it. Teams that don't, don't. ### Migrate from LabVIEW TestExec to Python URL: https://www.tofupilot.com/guides/how-to-migrate-from-labview-testexec-to-python-with-tofupilot A practical migration guide from LabVIEW TestExec to Python-based test automation with OpenHTF and TofuPilot, covering architecture mapping and code examples. # How to Migrate from LabVIEW TestExec to Python with TofuPilot LabVIEW TestExec served its purpose. But the licensing costs keep climbing, finding LabVIEW developers is getting harder, and your CI/CD pipeline doesn't speak G code. Python gives you the language, OpenHTF gives you the test framework, and TofuPilot gives you the data platform. Here's how to make the switch without losing your test coverage. ## Why Teams Migrate | Pain Point | LabVIEW TestExec | Python + TofuPilot | |---|---|---| | Licensing | Per-seat, annual renewal | Free and open source | | Developer pool | Shrinking, specialized | Large, growing | | Version control | Binary VIs, merge conflicts | Text files, standard Git | | CI/CD integration | Custom adapters needed | Native pytest/OpenHTF support | | Instrument drivers | NI ecosystem only | PyVISA, any SCPI instrument | | Data storage | Local TDM/TDMS files | Cloud database with API | ## Prerequisites - Python 3.8+ installed - `pip install openhtf tofupilot pyvisa` - Access to your existing LabVIEW test sequences and specs - Instrument documentation (SCPI command references) ## Step 1: Map Your TestExec Architecture LabVIEW TestExec and OpenHTF share similar concepts with different names: | LabVIEW TestExec | OpenHTF + TofuPilot | Notes | |---|---|---| | Test Sequence | `htf.Test()` | Top-level test container | | Test Step | Phase function | Decorated Python function | | Limit | `htf.Measurement` validator | `.in_range()`, `.at_most()`, etc. | | Step Result | Measurement value | Stored with pass/fail status | | Sequence File (.seq) | Python script (.py) | Version-controlled text | | Operator Interface | TofuPilot station UI | Web-based, no LabVIEW runtime | | Report | TofuPilot dashboard | Automatic, real-time | | Station Global | Module-level config or env var | Standard Python patterns | ## Step 2: Convert Test Steps to Python Phases A typical LabVIEW TestExec step that reads a voltage becomes a Python phase: **Before (LabVIEW TestExec pseudo-code):** ``` Step: "Read 5V Rail" Type: Numeric Limit Test Instrument: DMM_1 (NI PXI-4072) Command: Measure DC Voltage Low Limit: 4.95 High Limit: 5.05 Units: V ``` **After (Python with OpenHTF):** ```python filename="power_test.py" import openhtf as htf from openhtf.util import units import pyvisa @htf.measures( htf.Measurement("voltage_5v_rail") .with_units(units.VOLT) .in_range(4.95, 5.05) .doc("5V rail output voltage"), ) def read_5v_rail(test, dmm): """Read and validate 5V rail voltage.""" voltage = float(dmm.query("MEAS:VOLT:DC?")) test.measurements.voltage_5v_rail = voltage ``` The pattern is always the same: 1. Define measurements with limits (replaces the Limit columns) 2. Write instrument control in the phase body (replaces the VI) 3. Assign measured values (replaces the step result) ## Step 3: Handle Instrument Connections LabVIEW TestExec uses NI's instrument driver architecture. Python uses PyVISA, which works with NI, Keysight, Rigol, and any SCPI-compatible instrument. ```python filename="instruments.py" import pyvisa def connect_instruments(): """Replace TestExec station globals with PyVISA connections.""" rm = pyvisa.ResourceManager() instruments = { "dmm": rm.open_resource("TCPIP::192.168.1.10::INSTR"), "psu": rm.open_resource("TCPIP::192.168.1.11::INSTR"), "scope": rm.open_resource("USB0::0x0957::0x1796::INSTR"), } # Configure instruments instruments["dmm"].timeout = 5000 instruments["psu"].write("*RST") return instruments ``` If you're using NI PXI instruments, the NI-VISA backend works with PyVISA. Your GPIB, USB, and TCP/IP instruments work without any NI software. ## Step 4: Convert Sequence Flow Control LabVIEW TestExec has built-in flow control (preconditions, post-actions, branching). In OpenHTF, you use standard Python: **Conditional execution:** ```python filename="conditional_test.py" import openhtf as htf @htf.measures( htf.Measurement("board_variant") .with_allowed_values("A", "B", "C"), ) def detect_variant(test): """Read board variant from EEPROM or resistor divider.""" variant = read_board_variant() test.measurements.board_variant = variant test.state["variant"] = variant @htf.measures( htf.Measurement("bluetooth_rssi") .with_units(units.DECIBEL) .at_least(-70), ) def bluetooth_test(test): """Only runs on variant B (Bluetooth-equipped boards).""" if test.state.get("variant") != "B": return # Skip for non-BT variants rssi = scan_bluetooth() test.measurements.bluetooth_rssi = rssi ``` **Setup and teardown:** ```python filename="setup_teardown.py" import openhtf as htf @htf.PhaseOptions(timeout_s=10) def setup_fixture(test, fixture_controller): """Replaces TestExec Setup step group.""" fixture_controller.clamp() fixture_controller.connect_probes() @htf.PhaseOptions(run_if=lambda: True) # Always runs def teardown_fixture(test, fixture_controller): """Replaces TestExec Cleanup step group. Runs even on failure.""" fixture_controller.release() fixture_controller.disconnect_probes() ``` ## Step 5: Set Up Data Collection LabVIEW TestExec stores results in local files (TDM, TDMS, XML). TofuPilot stores everything in the cloud with full traceability. ```python filename="main_test.py" import openhtf as htf from tofupilot import TofuPilotClient def main(): instruments = connect_instruments() test = htf.Test( setup_fixture, detect_variant, read_5v_rail, bluetooth_test, teardown_fixture, phase_kwargs={ "dmm": instruments["dmm"], "psu": instruments["psu"], }, ) test.add_output_callbacks( TofuPilotClient().as_openhtf_callback( procedure_id="pcba-fct-v3", procedure_name="PCBA FCT (migrated from TestExec)", ) ) test.execute(test_start=htf.PhaseDescriptor.wrap( lambda test: setattr(test, "dut_id", input("Scan serial: ")) )) if __name__ == "__main__": main() ``` ## Migration Strategy Don't rewrite everything at once. Migrate station by station: 1. **Pick one test station** with the simplest sequence 2. **Document the existing test spec** (steps, limits, instruments, flow) 3. **Write the Python equivalent** following the patterns above 4. **Run both systems in parallel** for one production batch 5. **Compare results** to verify equivalence 6. **Cut over** when results match 7. **Move to the next station** Keep your LabVIEW TestExec sequences as reference documentation. The Python scripts replace them, but having the original spec helps during validation. ## Common Gotchas | Issue | Solution | |---|---| | NI drivers need LabVIEW runtime | PyVISA with NI-VISA backend works without LabVIEW | | TestExec parallel step groups | Use Python `threading` or `asyncio` | | TestExec callbacks (on fail, on pass) | OpenHTF phase options and output callbacks | | Station model/serial tracking | TofuPilot station configuration properties | | TestExec report generation | TofuPilot generates reports automatically | | Operator interface buttons | TofuPilot's web-based station UI or custom Tkinter | The hardest part isn't the code. It's validating that the new tests catch the same failures as the old ones. Run both systems in parallel until you're confident. ### How to Reduce Retesting Costs with TofuPilot URL: https://www.tofupilot.com/guides/how-to-reduce-retesting-costs-with-tofupilot Learn why retesting wastes 10-30% of test capacity, how to fix root causes with better OpenHTF patterns, and how TofuPilot tracks retest rates. Retesting eats 10-30% of test station capacity on a typical electronics production line. Every retest cycle burns station time, operator attention, and fixture wear, all without producing a single new unit. Cutting your retest rate is one of the fastest ways to increase throughput without buying more equipment. ## Why Retesting Is So Expensive The direct cost of a retest seems small: run the test again, maybe it passes. But the hidden costs add up fast. **Station throughput drops.** A station with 15% retest rate effectively loses 15% of its capacity. For a station running 500 units per shift, that's 75 slots wasted on units you already tested once. **Rework creates quality risk.** Every rework cycle (desolder, replace, resolder) introduces thermal stress, pad damage, and handling risk. Reworked units fail at higher rates downstream. **Data gets noisy.** When operators retest units without tracking serial numbers, your yield metrics become unreliable. You can't tell whether FPY is 85% or 95% because first attempts and retests are mixed together. ## Root Causes of Excessive Retesting Before you can fix retesting, you need to know why it's happening. The causes usually fall into four categories. ### Loose or Missing Test Limits Tests without proper limits can't distinguish real failures from marginal passes. If a voltage rail is specified at 3.3V +/- 5%, but your test has no limits, operators make judgment calls about what "looks okay." ### Flaky Test Fixtures Contact resistance on pogo pins, worn alignment features, and intermittent cable connections cause random failures. The unit is fine. The fixture isn't. Operators know this, so they retest. ### Operator-Initiated Re-runs Without clear pass/fail criteria, operators develop habits: "if it fails on voltage, just run it again." This masks both fixture problems and real defects. ### Environmental Sensitivity Temperature, humidity, and power supply variation can push measurements across limits. If your test runs differently at 8 AM versus 2 PM, you have an environment problem. ## OpenHTF Patterns That Reduce Retesting Good test code prevents unnecessary retests by catching fixture issues early and setting proper limits. ### Validate the Fixture Before Testing the DUT Add a fixture validation phase at the start of your test sequence. If the fixture fails, the run aborts before logging a false failure against the unit. ```python filename="test_with_fixture_check.py" import openhtf as htf from openhtf.util import units from tofupilot.openhtf import TofuPilot @htf.measures( htf.Measurement("fixture_contact_resistance") .with_units(units.OHM) .in_range(maximum=0.5), htf.Measurement("fixture_supply_voltage") .with_units(units.VOLT) .in_range(minimum=11.8, maximum=12.2), ) def validate_fixture(test): # Check pogo pin contact and power supply before DUT test test.measurements.fixture_contact_resistance = 0.12 test.measurements.fixture_supply_voltage = 12.01 @htf.measures( htf.Measurement("output_voltage") .with_units(units.VOLT) .in_range(minimum=3.135, maximum=3.465), htf.Measurement("ripple_mV") .in_range(maximum=50), ) def test_power_output(test): test.measurements.output_voltage = 3.29 test.measurements.ripple_mV = 18 @htf.measures( htf.Measurement("signal_integrity_dB") .in_range(minimum=-3.0, maximum=3.0), ) def test_signal_path(test): test.measurements.signal_integrity_dB = 0.4 def main(): test = htf.Test(validate_fixture, test_power_output, test_signal_path) with TofuPilot(test): test.execute(test_start=lambda: "SN-2026-01105") if __name__ == "__main__": main() ``` The `validate_fixture` phase runs first. If contact resistance is too high or the supply voltage is out of range, the test fails immediately. The failure is attributed to the fixture, not the DUT, and no false failure gets recorded against the serial number. ### Set Measurement Limits Based on Process Capability Don't guess limits. Set them from your Cpk data. If your process produces output voltage with a mean of 3.30V and a standard deviation of 0.03V, your 3-sigma range is 3.21V to 3.39V. Your test limits should be wider than 3-sigma (to avoid false failures) but within spec (to catch real defects). A good starting point is the spec limits from the component datasheet. ```python filename="calibrated_limits.py" import openhtf as htf from openhtf.util import units # Limits derived from Cpk study: spec is 3.0-3.6V, process sigma is 0.03V # Test limits set at spec boundaries to catch real defects @htf.measures( htf.Measurement("regulated_output") .with_units(units.VOLT) .in_range(minimum=3.0, maximum=3.6), htf.Measurement("load_regulation_pct") .in_range(minimum=-2.0, maximum=2.0), htf.Measurement("thermal_shutdown_temp") .with_units(units.DEGREE_CELSIUS) .in_range(minimum=145, maximum=155), ) def test_regulator(test): test.measurements.regulated_output = 3.31 test.measurements.load_regulation_pct = 0.8 test.measurements.thermal_shutdown_temp = 150 ``` ## How TofuPilot Tracks Retest Rates TofuPilot links every test run to a serial number, so it knows when a unit is being tested for the second, third, or tenth time. The dashboard shows you: - **Retest rate by station.** If one station has 3x the retest rate of another running the same test, the station needs maintenance. - **Unit history.** For any serial number, see every test attempt in sequence. Spot patterns like "fails once, passes on immediate retest" (fixture issue) versus "fails repeatedly on the same measurement" (real defect). - **FPY vs. output yield gap.** A large gap between FPY and output yield means you're relying heavily on retesting to hit your yield target. That gap is your retest cost, made visible. - **Failure Pareto.** See which test phases cause the most failures. If one phase accounts for 60% of retests, fixing that phase gives you the biggest return. ## A Practical Reduction Playbook 1. **Measure your current retest rate.** Check TofuPilot's FPY dashboard. The gap between FPY and output yield is your retest cost. 2. **Identify the top failure phase.** Use the failure Pareto chart. Focus on the single biggest contributor first. 3. **Classify failures.** For the top phase, check whether failures are fixture-related (pass on immediate retest) or DUT-related (consistent failure). 4. **Fix fixtures first.** Fixture failures are the cheapest to fix. Replace worn pogo pins, tighten alignment, and add fixture validation phases. 5. **Review test limits.** Use measurement histograms and Cpk charts in TofuPilot to check whether limits match process capability. Tighten limits that are too loose. Widen limits where Cpk shows the process can't hit them. 6. **Repeat monthly.** Retest rate creeps up as fixtures wear and process conditions change. Make it a recurring review. ### How to Build a Test Sequencer with TofuPilot URL: https://www.tofupilot.com/guides/how-to-build-a-test-sequencer-with-python-and-tofupilot Learn how to build a test sequencer with OpenHTF using phase ordering, skip logic, PhaseGroups, multi-SKU sequences, and TofuPilot result logging. A test sequencer runs ordered phases, decides which to skip, collects operator input, and records the outcome. This guide shows how to build one with OpenHTF and log structured results to TofuPilot. ## Prerequisites - Python 3.8+ - TofuPilot account and API key - OpenHTF installed: `pip install openhtf tofupilot` ## What a Test Sequencer Does A sequencer is more than a list of test steps. It controls: | Responsibility | Example | |---|---| | Phase ordering | Power-on before functional tests | | Skip logic | Skip RF calibration if hardware variant lacks antenna | | Operator interaction | Prompt for serial number, confirm visual inspection | | Result aggregation | Single pass/fail outcome from all phases | | Teardown | Power off even when a phase fails | OpenHTF maps directly to these responsibilities. Each phase is a Python function. The test object sequences them, evaluates measurements, and streams results to TofuPilot. ## Step 1: Define Phases Each phase is a decorated function. Add measurements to capture numeric results with limits. ```python filename="sequencer.py" import openhtf as htf from openhtf.util import units @htf.measures( htf.Measurement('supply_voltage') .in_range(minimum=4.75, maximum=5.25) .with_units(units.VOLT) ) def power_on_test(test): voltage = read_supply_voltage() test.measurements.supply_voltage = voltage @htf.measures( htf.Measurement('loop_back_result').equals(True) ) def uart_loopback_test(test): result = send_uart_loopback() test.measurements.loop_back_result = result @htf.measures( htf.Measurement('output_current') .in_range(minimum=0.9, maximum=1.1) .with_units(units.AMPERE) ) def load_test(test): current = measure_output_current() test.measurements.output_current = current ``` ## Step 2: Use PhaseGroups for Setup and Teardown `PhaseGroup` guarantees teardown runs even when a test phase fails. This is critical for hardware tests that hold relays, power supplies, or fixtures. ```python filename="sequencer.py" from openhtf import PhaseGroup def power_on(test): enable_power_supply() test.logger.info('Power supply enabled') def power_off(test): # Runs even if functional_tests fails disable_power_supply() test.logger.info('Power supply disabled') # Build the group: setup -> main -> teardown test_group = PhaseGroup( setup=[power_on], main=[power_on_test, uart_loopback_test, load_test], teardown=[power_off], ) ``` `teardown` phases run regardless of outcome. `setup` failures skip `main` but still run `teardown`. ## Step 3: Inject Plugs for Hardware Access Plugs encapsulate instrument or fixture communication. Inject them with `@htf.plug`. ```python filename="plugs/supply.py" import openhtf as htf class PowerSupplyPlug(htf.plugs.BasePlug): def setUp(self): self._conn = open_supply_connection() def read_voltage(self): return self._conn.query('MEAS:VOLT?') def tearDown(self): self._conn.close() ``` ```python filename="sequencer.py" from openhtf.util import units from plugs.supply import PowerSupplyPlug @htf.plug(supply=PowerSupplyPlug) @htf.measures( htf.Measurement('supply_voltage') .in_range(minimum=4.75, maximum=5.25) .with_units(units.VOLT) ) def power_on_test(test, supply): test.measurements.supply_voltage = supply.read_voltage() ``` Note the `@htf.plug(name=PlugClass)` form. Don't use type hints for plug injection. ## Step 4: Add Skip Logic for Conditional Execution Return `htf.PhaseResult.SKIP` to bypass a phase based on DUT variant. ```python filename="sequencer.py" import openhtf as htf @htf.measures( htf.Measurement('rf_output_power') .in_range(minimum=18.0, maximum=22.0) ) def rf_calibration_test(test): if not has_rf_module(): test.logger.info('No RF module detected, skipping RF calibration') return htf.PhaseResult.SKIP power = measure_rf_output() test.measurements.rf_output_power = power ``` Keep skip decisions data-driven rather than hardcoded. Read hardware configuration in an earlier phase or from a config file. ## Step 5: Build Multi-SKU Sequences Different product variants need different phase sets. Compose sequences dynamically from a SKU map. ```python filename="sequencer.py" from openhtf import PhaseGroup # Base phases run on every SKU BASE_PHASES = [power_on_test, uart_loopback_test] # Extra phases per SKU SKU_PHASES = { 'MODEL_A': [load_test], 'MODEL_B': [load_test, rf_calibration_test], 'MODEL_C': [], # base only } def build_sequence(sku: str): extra = SKU_PHASES.get(sku, []) return PhaseGroup( setup=[power_on], main=BASE_PHASES + extra, teardown=[power_off], ) ``` ## Step 6: Log Results to TofuPilot Wrap the test execution with `TofuPilot`. It captures every phase, measurement, and outcome. ```python filename="sequencer.py" import openhtf as htf from tofupilot.openhtf import TofuPilot def main(sku: str = 'MODEL_A'): sequence = build_sequence(sku) test = htf.Test( sequence, test_name=f'Functional Test {sku}', ) with TofuPilot(test): test.execute(test_start=lambda: input('Enter serial number: ')) if __name__ == '__main__': import sys sku = sys.argv[1] if len(sys.argv) > 1 else 'MODEL_A' main(sku) ``` ## Phase Outcome Reference | Return value | Effect | |---|---| | `htf.PhaseResult.CONTINUE` | Phase passed, continue sequence | | `htf.PhaseResult.SKIP` | Phase skipped, continue sequence | | `htf.PhaseResult.STOP` | Phase failed, stop sequence (run teardown) | | Measurement out of range | Phase fails automatically | | Unhandled exception | Phase fails, teardown runs | ## Full Example ```python filename="sequencer.py" import sys import openhtf as htf from openhtf import PhaseGroup from openhtf.util import units from tofupilot.openhtf import TofuPilot # -- Phases -- @htf.measures( htf.Measurement('supply_voltage') .in_range(minimum=4.75, maximum=5.25) .with_units(units.VOLT) ) def power_on_test(test): test.measurements.supply_voltage = read_supply_voltage() @htf.measures( htf.Measurement('uart_loopback').equals(True) ) def uart_loopback_test(test): test.measurements.uart_loopback = send_uart_loopback() @htf.measures( htf.Measurement('output_current') .in_range(minimum=0.9, maximum=1.1) .with_units(units.AMPERE) ) def load_test(test): test.measurements.output_current = measure_output_current() @htf.measures( htf.Measurement('rf_output_power') .in_range(minimum=18.0, maximum=22.0) ) def rf_calibration_test(test): if not has_rf_module(): return htf.PhaseResult.SKIP test.measurements.rf_output_power = measure_rf_output() def power_on(test): enable_power_supply() def power_off(test): disable_power_supply() # -- Sequence builder -- BASE_PHASES = [power_on_test, uart_loopback_test] SKU_PHASES = { 'MODEL_A': [load_test], 'MODEL_B': [load_test, rf_calibration_test], } def build_sequence(sku): return PhaseGroup( setup=[power_on], main=BASE_PHASES + SKU_PHASES.get(sku, []), teardown=[power_off], ) # -- Entry point -- def main(): sku = sys.argv[1] if len(sys.argv) > 1 else 'MODEL_A' test = htf.Test(build_sequence(sku), test_name=f'Functional Test {sku}') with TofuPilot(test): test.execute(test_start=lambda: input('Enter serial number: ')) if __name__ == '__main__': main() ``` ### Hardware Test Observability with TofuPilot URL: https://www.tofupilot.com/guides/hardware-test-observability-with-tofupilot Learn how to build full observability into your hardware test operations using TofuPilot's dashboards, alerts, and analytics. # Hardware Test Observability with TofuPilot Software teams have Datadog and Grafana. Hardware test teams have... shared drives full of CSV files. Test observability means having the same real-time visibility into your hardware test operations that software teams take for granted. ## What Test Observability Means for Hardware Observability in software is about understanding system behavior from its outputs (logs, metrics, traces). Hardware test observability applies the same principle: understand your production quality from test outputs (measurements, pass/fail rates, cycle times). | Software Observability | Hardware Test Observability | |----------------------|---------------------------| | Error rates | First-pass yield (FPY) | | Latency metrics | Test cycle time | | Log aggregation | Test result centralization | | Alerting on anomalies | Yield drop notifications | | Distributed tracing | Unit traceability across stations | ## The Three Pillars of Hardware Test Observability ### 1. Metrics Quantitative data about your test operations. - **First-pass yield (FPY)** per procedure, station, and time period - **Measurement distributions** for every parameter you test - **Cpk/Ppk** process capability indices - **Test cycle time** and throughput - **Failure pareto** showing top failure modes TofuPilot computes these automatically from your test data. No spreadsheet formulas, no manual aggregation. ### 2. Logs Every test run is a log entry. TofuPilot stores: - Full measurement data with limits and units - Pass/fail status per step and per run - Timestamps, station identifiers, operator info - Attachments (waveforms, images, log files) - Unit metadata (serial number, revision, batch) Unlike CSV files on a shared drive, these logs are indexed, searchable, and queryable through the dashboard and API. ### 3. Traces A unit's journey through your test process is a trace. TofuPilot links all test runs for a given serial number, showing: - Which tests the unit has passed - Which station ran each test - When each test was executed - The complete measurement history This is critical for units that go through multiple test stages (ICT, functional, burn-in, final test). The full trace tells you everything that happened to a unit from first test to ship. ## Building Observability with TofuPilot ### Step 1: Centralize All Test Data Every test station should push results to TofuPilot. This eliminates data silos. ```python filename="station_setup.py" from tofupilot import TofuPilotClient # Same client works for every station client = TofuPilotClient() # Every run is automatically indexed by procedure, station, and unit client.create_run( procedure_id="FINAL-TEST-V2", unit_under_test={"serial_number": "UNIT-8847"}, run_passed=True, steps=[...], ) ``` ### Step 2: Use Dashboards for Real-Time Visibility TofuPilot's procedure dashboard shows live metrics: - Current FPY with trend line - Measurement distributions with limit overlays - Recent run history with pass/fail status - Failure mode breakdown Pin the dashboards on a monitor near the production line. When yield drops, everyone sees it immediately. ### Step 3: Set Up Yield Monitoring Track FPY over time to catch regressions early. A yield drop from 98% to 94% over a week is easy to miss in daily noise but obvious in a trend chart. ### Step 4: Enable Unit Traceability When a field return comes in, search by serial number in TofuPilot. Every test the unit ever ran is there: measurements, pass/fail status, timestamps, and which station tested it. No digging through old files. ## Observability vs. Reporting | Reporting | Observability | |-----------|--------------| | Backward-looking | Real-time | | Manual (someone pulls a report) | Automatic (dashboards update live) | | Aggregated (weekly/monthly summaries) | Granular (every run, every measurement) | | Static (PDF/Excel) | Interactive (filter, drill down, compare) | Reporting tells you what happened last month. Observability tells you what's happening right now. ## Common Observability Wins **Catching a yield drop in hours, not weeks.** Without observability, yield problems surface in weekly quality reviews. With TofuPilot's live dashboards, engineers see the drop the same day it starts. **Finding station-specific issues.** One station has 91% FPY while the others run at 98%. Without centralized data, this is invisible because each station has its own files. TofuPilot's station comparison makes it obvious. **Reducing debug time for field returns.** Customer reports a failure. Instead of guessing what happened during production, pull the unit's full test history in seconds. Every measurement, every step, every station. **Proving compliance with data.** When auditors ask for test records, point them to TofuPilot. Structured, timestamped, immutable records replace binders full of printouts. ### How to Build a Test Panel GUI with TofuPilot URL: https://www.tofupilot.com/guides/how-to-build-a-python-test-panel-gui-for-instrument-control Learn how to build a tkinter-based test panel GUI that controls instruments and runs OpenHTF tests with TofuPilot logging. Building a custom operator panel lets your test technicians run tests without touching the command line. This guide shows how to create a tkinter-based GUI that controls instruments, collects serial numbers, and runs OpenHTF tests with TofuPilot logging. ## Prerequisites - Python 3.8+ - OpenHTF installed (`pip install openhtf`) - TofuPilot Python SDK installed (`pip install tofupilot`) - A TofuPilot API key configured ## Step 1: Create the Main Panel Layout Start with a basic tkinter window that has a serial number entry, instrument controls, and a test execution area. ```python filename="panel.py" import tkinter as tk from tkinter import ttk import threading class TestPanel: def __init__(self, root): self.root = root self.root.title("Test Panel") self.root.geometry("480x520") self.root.resizable(False, False) self.instrument_connected = False self._build_instrument_frame() self._build_serial_frame() self._build_control_frame() self._build_status_frame() def _build_instrument_frame(self): frame = ttk.LabelFrame(self.root, text="Instrument", padding=10) frame.pack(fill="x", padx=10, pady=(10, 5)) self.connect_btn = ttk.Button( frame, text="Connect", command=self._connect_instrument ) self.connect_btn.pack(side="left", padx=(0, 5)) self.disconnect_btn = ttk.Button( frame, text="Disconnect", command=self._disconnect_instrument, state="disabled" ) self.disconnect_btn.pack(side="left") self.instr_status = ttk.Label(frame, text="Disconnected", foreground="gray") self.instr_status.pack(side="right") def _build_serial_frame(self): frame = ttk.LabelFrame(self.root, text="Unit Under Test", padding=10) frame.pack(fill="x", padx=10, pady=5) ttk.Label(frame, text="Serial Number:").pack(anchor="w") self.serial_var = tk.StringVar() self.serial_entry = ttk.Entry(frame, textvariable=self.serial_var, width=30) self.serial_entry.pack(fill="x", pady=(2, 0)) def _build_control_frame(self): frame = ttk.Frame(self.root, padding=10) frame.pack(fill="x", padx=10) self.start_btn = ttk.Button( frame, text="Start Test", command=self._start_test, state="disabled" ) self.start_btn.pack(fill="x") def _build_status_frame(self): frame = ttk.LabelFrame(self.root, text="Status", padding=10) frame.pack(fill="both", expand=True, padx=10, pady=(5, 10)) self.result_label = ttk.Label( frame, text="Idle", font=("Helvetica", 18, "bold"), anchor="center" ) self.result_label.pack(pady=(0, 10)) self.log_text = tk.Text(frame, height=10, state="disabled", font=("Courier", 10)) self.log_text.pack(fill="both", expand=True) def _log(self, message): self.log_text.configure(state="normal") self.log_text.insert("end", message + "\n") self.log_text.see("end") self.log_text.configure(state="disabled") def _set_result(self, text, color): self.result_label.configure(text=text, foreground=color) def _connect_instrument(self): self._log("Connecting to instrument...") self.instrument_connected = True self.instr_status.configure(text="Connected", foreground="green") self.connect_btn.configure(state="disabled") self.disconnect_btn.configure(state="normal") self.start_btn.configure(state="normal") self._log("Instrument connected.") def _disconnect_instrument(self): self._log("Disconnecting instrument...") self.instrument_connected = False self.instr_status.configure(text="Disconnected", foreground="gray") self.connect_btn.configure(state="normal") self.disconnect_btn.configure(state="disabled") self.start_btn.configure(state="disabled") self._log("Instrument disconnected.") def _start_test(self): serial = self.serial_var.get().strip() if not serial: self._log("Error: enter a serial number.") return self.start_btn.configure(state="disabled") self.serial_entry.configure(state="disabled") self._set_result("Running...", "orange") self._log(f"Starting test for {serial}...") thread = threading.Thread(target=self._run_test, args=(serial,), daemon=True) thread.start() def _run_test(self, serial): # Placeholder — see Step 4 for full implementation pass if __name__ == "__main__": root = tk.Tk() app = TestPanel(root) root.mainloop() ``` This gives you a clean panel with instrument connect/disconnect buttons, a serial number field, and a status area that shows logs and pass/fail results. ## Step 2: Add an Instrument Plug Create an OpenHTF plug that wraps your instrument communication. This example simulates a power supply, but you'd replace the internals with your actual SCPI or serial commands. ```python filename="instrument_plug.py" import time import openhtf as htf class PowerSupplyPlug(htf.plugs.BasePlug): """Plug for controlling a bench power supply.""" ADDRESS = "GPIB0::5::INSTR" def setUp(self): self._connected = False self._voltage = 0.0 # Replace with real instrument connection (e.g., pyvisa) self._connected = True def set_voltage(self, voltage): if not self._connected: raise RuntimeError("Power supply not connected") self._voltage = voltage time.sleep(0.3) def measure_current(self): if not self._connected: raise RuntimeError("Power supply not connected") time.sleep(0.2) return 0.125 def tearDown(self): if self._connected: self.set_voltage(0.0) self._connected = False ``` The `tearDown` method runs automatically when the test finishes, so the instrument always returns to a safe state. ## Step 3: Define OpenHTF Test Phases Write test phases that use the instrument plug to take measurements. Note how plug injection uses the `@htf.plug` decorator. ```python filename="test_phases.py" import openhtf as htf from openhtf.util import units from instrument_plug import PowerSupplyPlug @htf.plug(psu=PowerSupplyPlug) @htf.measures( htf.Measurement("supply_voltage_setpoint").with_units(units.VOLT), htf.Measurement("quiescent_current") .in_range(minimum=0.05, maximum=0.25) .with_units(units.AMPERE), ) def phase_quiescent_current(test, psu): """Measure quiescent current at nominal voltage.""" psu.set_voltage(3.3) test.measurements.supply_voltage_setpoint = 3.3 current = psu.measure_current() test.measurements.quiescent_current = current @htf.plug(psu=PowerSupplyPlug) @htf.measures( htf.Measurement("overcurrent_detected").equals(True), ) def phase_overcurrent_protection(test, psu): """Verify the DUT triggers overcurrent protection at high voltage.""" psu.set_voltage(5.5) test.measurements.overcurrent_detected = True ``` Each phase focuses on one thing. The measurement validators (`.in_range`, `.equals`) tell OpenHTF what counts as a pass or fail. ## Step 4: Wire the GUI to OpenHTF Execution Now connect the panel's `_run_test` method to actually execute the OpenHTF test with TofuPilot logging. ```python filename="panel_with_test.py" import tkinter as tk from tkinter import ttk import threading import openhtf as htf from tofupilot.openhtf import TofuPilot from test_phases import phase_quiescent_current, phase_overcurrent_protection class TestPanel: def __init__(self, root): self.root = root self.root.title("Test Panel") self.root.geometry("480x520") self.root.resizable(False, False) self.instrument_connected = False self._build_instrument_frame() self._build_serial_frame() self._build_control_frame() self._build_status_frame() def _build_instrument_frame(self): frame = ttk.LabelFrame(self.root, text="Instrument", padding=10) frame.pack(fill="x", padx=10, pady=(10, 5)) self.connect_btn = ttk.Button( frame, text="Connect", command=self._connect_instrument ) self.connect_btn.pack(side="left", padx=(0, 5)) self.disconnect_btn = ttk.Button( frame, text="Disconnect", command=self._disconnect_instrument, state="disabled" ) self.disconnect_btn.pack(side="left") self.instr_status = ttk.Label(frame, text="Disconnected", foreground="gray") self.instr_status.pack(side="right") def _build_serial_frame(self): frame = ttk.LabelFrame(self.root, text="Unit Under Test", padding=10) frame.pack(fill="x", padx=10, pady=5) ttk.Label(frame, text="Serial Number:").pack(anchor="w") self.serial_var = tk.StringVar() self.serial_entry = ttk.Entry(frame, textvariable=self.serial_var, width=30) self.serial_entry.pack(fill="x", pady=(2, 0)) def _build_control_frame(self): frame = ttk.Frame(self.root, padding=10) frame.pack(fill="x", padx=10) self.start_btn = ttk.Button( frame, text="Start Test", command=self._start_test, state="disabled" ) self.start_btn.pack(fill="x") def _build_status_frame(self): frame = ttk.LabelFrame(self.root, text="Status", padding=10) frame.pack(fill="both", expand=True, padx=10, pady=(5, 10)) self.result_label = ttk.Label( frame, text="Idle", font=("Helvetica", 18, "bold"), anchor="center" ) self.result_label.pack(pady=(0, 10)) self.log_text = tk.Text(frame, height=10, state="disabled", font=("Courier", 10)) self.log_text.pack(fill="both", expand=True) def _log(self, message): self.log_text.configure(state="normal") self.log_text.insert("end", message + "\n") self.log_text.see("end") self.log_text.configure(state="disabled") def _set_result(self, text, color): self.result_label.configure(text=text, foreground=color) def _connect_instrument(self): self._log("Connecting to instrument...") self.instrument_connected = True self.instr_status.configure(text="Connected", foreground="green") self.connect_btn.configure(state="disabled") self.disconnect_btn.configure(state="normal") self.start_btn.configure(state="normal") self._log("Instrument connected.") def _disconnect_instrument(self): self._log("Disconnecting instrument...") self.instrument_connected = False self.instr_status.configure(text="Disconnected", foreground="gray") self.connect_btn.configure(state="normal") self.disconnect_btn.configure(state="disabled") self.start_btn.configure(state="disabled") self._log("Instrument disconnected.") def _start_test(self): serial = self.serial_var.get().strip() if not serial: self._log("Error: enter a serial number.") return self.start_btn.configure(state="disabled") self.serial_entry.configure(state="disabled") self._set_result("Running...", "orange") self._log(f"Starting test for {serial}...") thread = threading.Thread(target=self._run_test, args=(serial,), daemon=True) thread.start() def _run_test(self, serial): try: test = htf.Test( phase_quiescent_current, phase_overcurrent_protection, procedure_id="PCB-FUNC-001", part_number="PCB-2400", ) with TofuPilot(test): test_record = test.execute(test_start=lambda: serial) outcome = test_record.outcome.name if outcome == "PASS": self.root.after(0, self._set_result, "PASS", "green") self.root.after(0, self._log, f"Test PASSED for {serial}.") else: self.root.after(0, self._set_result, "FAIL", "red") self.root.after(0, self._log, f"Test FAILED for {serial}.") except Exception as e: self.root.after(0, self._set_result, "ERROR", "red") self.root.after(0, self._log, f"Error: {e}") finally: self.root.after(0, self._reset_controls) def _reset_controls(self): self.serial_entry.configure(state="normal") self.serial_var.set("") if self.instrument_connected: self.start_btn.configure(state="normal") if __name__ == "__main__": root = tk.Tk() app = TestPanel(root) root.mainloop() ``` A few things to note here: - `test.execute` runs synchronously in the background thread, so the GUI stays responsive. - `root.after(0, ...)` schedules GUI updates on the main thread. Tkinter isn't thread-safe, so you can't call widget methods directly from a worker thread. - `TofuPilot(test)` wraps execution and automatically uploads the test record, including all measurements and outcomes. ### How to Reduce Field Returns with TofuPilot URL: https://www.tofupilot.com/guides/how-to-reduce-field-returns-with-tofupilot Learn how to catch marginal units before they ship by adding margin tests and stress phases in OpenHTF, then correlating field failures in TofuPilot. Units that pass production testing but fail in the field are expensive. Every field return costs 10 to 100 times more than catching the same defect on the line. The fix starts with better test coverage and tighter analysis of the data you're already collecting. TofuPilot gives you the tools to spot marginal units before they ship. ## Why Units Pass Tests but Fail in the Field Three root causes account for most field returns: **Insufficient test coverage.** Your production test checks 80% of the functionality, but the 20% you skip includes the failure mode customers hit. If you don't test the sleep/wake cycle, you won't catch the firmware bug that corrupts EEPROM after 1,000 transitions. **Test limits too loose.** A unit measures 4.65V against a 4.5V to 5.5V spec. It passes, but it's drifting toward the edge. Temperature shifts in the field push it out of range. Your test caught nothing because the limits matched the datasheet, not real-world conditions. **Environmental conditions not tested.** Production tests run at 25C on a bench. The product operates at 0C to 50C in a warehouse with vibration. Parameters that look solid at room temperature fall apart under stress. ## Add Margin Testing to Your OpenHTF Tests Margin testing uses tighter limits than the product specification. The idea is simple: if a unit can't pass with headroom, it's more likely to fail in the field. ```python filename="margin_test.py" import openhtf as htf from openhtf.util import units from tofupilot.openhtf import TofuPilot @htf.measures( htf.Measurement("output_voltage") .in_range(minimum=4.8, maximum=5.2) .with_units(units.VOLT) .doc("Spec is 4.5-5.5V. Margin limits tightened to 4.8-5.2V."), htf.Measurement("ripple_voltage") .in_range(maximum=20) .with_units(units.MILLIVOLT) .doc("Spec allows 50mV. Margin limit set to 20mV."), htf.Measurement("quiescent_current") .in_range(maximum=0.000008) .with_units(units.AMPERE) .doc("Spec allows 15uA. Margin limit catches leaky units."), ) def margin_check(test): test.measurements.output_voltage = 5.03 test.measurements.ripple_voltage = 12.4 test.measurements.quiescent_current = 0.0000051 def main(): test = htf.Test(margin_check) with TofuPilot(test, procedure_name="PCBA Margin Test"): test.execute(test_start=lambda: "SN-20260312-042") if __name__ == "__main__": main() ``` Use the `.doc()` method to record both the spec limit and your margin limit. This makes it clear to anyone reviewing the data why a unit at 4.75V failed even though the datasheet says 4.5V is fine. ## Add Environmental Stress Phases For products that operate across a temperature range, add stress phases that test at the boundaries. You don't need a full thermal chamber for every unit. Even a brief hot/cold soak on a sample basis catches drift. ```python filename="stress_test.py" import openhtf as htf from openhtf.util import units from tofupilot.openhtf import TofuPilot @htf.measures( htf.Measurement("voltage_ambient") .in_range(minimum=4.8, maximum=5.2) .with_units(units.VOLT), ) def ambient_check(test): # Measure at room temperature baseline test.measurements.voltage_ambient = 5.01 @htf.measures( htf.Measurement("voltage_hot") .in_range(minimum=4.7, maximum=5.3) .with_units(units.VOLT), htf.Measurement("voltage_drift_hot") .in_range(maximum=100) .with_units(units.MILLIVOLT) .doc("Delta between 25C and 50C readings"), ) def hot_soak_check(test): # Measure after thermal soak at upper operating limit test.measurements.voltage_hot = 5.08 test.measurements.voltage_drift_hot = 70 @htf.measures( htf.Measurement("voltage_cold") .in_range(minimum=4.7, maximum=5.3) .with_units(units.VOLT), htf.Measurement("voltage_drift_cold") .in_range(maximum=100) .with_units(units.MILLIVOLT) .doc("Delta between 25C and 0C readings"), ) def cold_soak_check(test): # Measure after thermal soak at lower operating limit test.measurements.voltage_cold = 4.92 test.measurements.voltage_drift_cold = 90 def main(): test = htf.Test(ambient_check, hot_soak_check, cold_soak_check) with TofuPilot(test, procedure_name="PCBA Thermal Stress Test"): test.execute(test_start=lambda: "SN-20260312-042") if __name__ == "__main__": main() ``` Recording the drift as a separate measurement is key. A unit might pass both the hot and cold absolute checks but show 95mV of drift, which signals a component that's near its thermal limit. ## Correlate Field Failures Back to Production Data When a unit comes back from the field, look it up by serial number in TofuPilot. The unit history shows every test run, including all measurements recorded during production. Here's the workflow: 1. **Get the serial number** from the returned unit. 2. **Search in TofuPilot** to pull up the unit's full test history. 3. **Check the measurements** for the failure mode. If the field failure is "intermittent power loss," look at the production voltage and current measurements. 4. **Compare against passing units.** Was the returned unit near the edge of a limit? Did it show higher drift than units that survived? This comparison tells you whether your limits need tightening or your test coverage has gaps. ## Identify Marginal Units with Measurement Analytics TofuPilot's measurement analytics show you the distribution of every measurement across all units. This is where you catch systematic issues before they become field returns. **Measurement histograms** reveal bimodal distributions. If your voltage measurement shows two clusters instead of one normal curve, you likely have a component sourcing issue or a process variation. Units in the lower cluster are your field return candidates. **Cpk values** tell you how centered your process is within the limits. A Cpk below 1.33 means your process variation is too wide relative to your test limits. Even if every unit passes today, normal variation will push some units out of spec in the field. **Control charts** show measurement trends over time. A gradual drift in a measurement across production batches signals a process change. Catching the drift early lets you investigate before marginal units ship. ## Tighten Limits Based on Field Data Once you've correlated field returns with production measurements, update your test limits. The pattern is straightforward: | Situation | Action | |-----------|--------| | Returned units clustered near upper limit | Lower the upper margin limit | | Returned units show high thermal drift | Add or tighten drift measurement limit | | Failure mode not covered by any measurement | Add a new test phase for that parameter | | Returns correlate with a specific production batch | Investigate process change during that period | Don't guess at new limits. Use TofuPilot's measurement distribution data from your passing field population to set limits that exclude the tail where failures concentrate. ## Add Coverage for Missing Failure Modes Field returns often reveal failure modes your test doesn't cover at all. When that happens, add a new phase to your OpenHTF test. ```python filename="added_coverage.py" import openhtf as htf from openhtf.util import units from tofupilot.openhtf import TofuPilot @htf.measures( htf.Measurement("sleep_wake_cycles_passed") .in_range(minimum=100) .doc("Added after field returns showed EEPROM corruption after repeated sleep/wake."), htf.Measurement("eeprom_checksum_valid") .equals(True), ) def sleep_wake_stress(test): # Run 100 sleep/wake cycles and verify EEPROM integrity cycles_passed = 100 checksum_ok = True test.measurements.sleep_wake_cycles_passed = cycles_passed test.measurements.eeprom_checksum_valid = checksum_ok @htf.measures( htf.Measurement("output_voltage") .in_range(minimum=4.8, maximum=5.2) .with_units(units.VOLT), ) def functional_check(test): test.measurements.output_voltage = 5.02 def main(): test = htf.Test(sleep_wake_stress, functional_check) with TofuPilot(test, procedure_name="PCBA Functional Test v2"): test.execute(test_start=lambda: "SN-20260312-099") if __name__ == "__main__": main() ``` Document why you added the phase using `.doc()`. Six months from now, someone will ask why you're running 100 sleep/wake cycles on every unit. The answer is right there in the measurement metadata and visible in TofuPilot's run detail view. ## Build a Feedback Loop Reducing field returns isn't a one-time fix. It's a continuous loop: 1. **Ship units** with your current test coverage and limits. 2. **Track returns** by serial number. 3. **Correlate** return data with production measurements in TofuPilot. 4. **Tighten limits** or add test phases based on what you find. 5. **Monitor Cpk and FPY** in TofuPilot to confirm the changes reduce returns without over-rejecting good units. Each iteration makes your test suite more effective. TofuPilot's measurement analytics and unit history give you the data to make each decision evidence-based rather than guesswork. ### Cell Grading and Binning with TofuPilot URL: https://www.tofupilot.com/guides/cell-grading-and-binning-with-tofupilot Learn how to grade and bin battery cells based on test data using TofuPilot for capacity matching, impedance sorting, and production traceability. # Cell Grading and Binning with TofuPilot A battery pack is only as good as its weakest cell. Cell grading sorts production cells by measured performance so you can build packs from matched cells. TofuPilot stores every cell's test data and lets you query, filter, and grade cells based on actual measurements. ## Why Cell Grading Matters Cells from the same production line have natural variation. Capacity varies by 2-5%. Impedance varies by 10-20%. If you build a pack from unmatched cells: - The weakest cell limits pack capacity - Cells age at different rates, causing imbalance - BMS has to work harder to keep cells balanced - Pack lifetime is shorter than it needs to be Grading cells before assembly ensures packs perform consistently and last longer. ## Grading Criteria | Parameter | How it's measured | Why it matters | |-----------|------------------|---------------| | Capacity (Ah) | Full charge/discharge cycle | Determines pack energy | | Internal impedance (mohm) | AC impedance at 1kHz | Affects power delivery and heat | | Open-circuit voltage (V) | Rest voltage after formation | Indicates state of charge consistency | | Self-discharge rate (mV/day) | Voltage drop over 7-14 day rest | Catches internal micro-shorts | | Weight (g) | Scale measurement | Catches underfilled or overfilled cells | ## Setting Up Cell Grading in TofuPilot ### Step 1: Define the Grading Test Procedure ```python filename="cell_grading_test.py" from tofupilot import TofuPilotClient client = TofuPilotClient() def grade_cell(serial, capacity_ah, impedance_mohm, ocv_v, self_discharge_mv_day, weight_g): # Determine grade if capacity_ah >= 3.1 and impedance_mohm <= 30 and self_discharge_mv_day <= 0.5: grade = "A" passed = True elif capacity_ah >= 2.9 and impedance_mohm <= 40 and self_discharge_mv_day <= 1.0: grade = "B" passed = True elif capacity_ah >= 2.7 and impedance_mohm <= 50: grade = "C" passed = True else: grade = "REJECT" passed = False client.create_run( procedure_id="CELL-GRADING", unit_under_test={ "serial_number": serial, "part_number": "CELL-21700-NMC", }, run_passed=passed, steps=[{ "name": "Capacity Test", "step_type": "measurement", "status": capacity_ah >= 2.7, "measurements": [ {"name": "capacity_ah", "value": capacity_ah, "unit": "Ah", "limit_low": 2.7}, ], }, { "name": "Impedance Test", "step_type": "measurement", "status": impedance_mohm <= 50, "measurements": [ {"name": "impedance_mohm", "value": impedance_mohm, "unit": "mohm", "limit_high": 50}, ], }, { "name": "Self-Discharge", "step_type": "measurement", "status": self_discharge_mv_day <= 1.0, "measurements": [ {"name": "self_discharge_mv_day", "value": self_discharge_mv_day, "unit": "mV/day", "limit_high": 1.0}, {"name": "ocv_v", "value": ocv_v, "unit": "V", "limit_low": 3.5, "limit_high": 4.2}, ], }, { "name": "Physical", "step_type": "measurement", "status": 68.0 <= weight_g <= 72.0, "measurements": [ {"name": "weight_g", "value": weight_g, "unit": "g", "limit_low": 68.0, "limit_high": 72.0}, {"name": "grade", "value": grade, "unit": ""}, ], }], ) return grade ``` ### Step 2: Analyze Grade Distribution After grading a production batch, TofuPilot's dashboard shows the distribution: | Grade | Count | Percentage | Capacity range | Impedance range | |-------|-------|-----------|----------------|-----------------| | A | 850 | 85% | 3.10-3.25 Ah | 22-30 mohm | | B | 120 | 12% | 2.90-3.09 Ah | 31-40 mohm | | C | 20 | 2% | 2.70-2.89 Ah | 41-50 mohm | | REJECT | 10 | 1% | < 2.70 Ah | > 50 mohm | Track this distribution over time. If Grade A yield drops from 85% to 75%, the manufacturing process is drifting. ### Step 3: Pack Assembly with Matched Cells Query TofuPilot for Grade A cells with capacity within a tight window for pack assembly. ```python filename="pack_matching.py" from tofupilot import TofuPilotClient client = TofuPilotClient() # Get all Grade A cells from the latest batch runs = client.get_runs( procedure_id="CELL-GRADING", run_passed=True, limit=1000, ) # Filter for Grade A cells within a 50mAh capacity window grade_a_cells = [] for run in runs: for step in run.get("steps", []): for m in step.get("measurements", []): if m["name"] == "capacity_ah" and m["value"] >= 3.1: grade_a_cells.append({ "serial": run["unit_under_test"]["serial_number"], "capacity": m["value"], }) # Sort by capacity and group into matched sets grade_a_cells.sort(key=lambda c: c["capacity"]) # Create packs of 12 cells with capacity spread < 50mAh pack_size = 12 for i in range(0, len(grade_a_cells) - pack_size + 1, pack_size): pack = grade_a_cells[i:i + pack_size] spread = pack[-1]["capacity"] - pack[0]["capacity"] if spread <= 0.05: print(f"Pack: {[c['serial'] for c in pack]}, spread: {spread*1000:.0f} mAh") ``` ## Self-Discharge Screening Self-discharge is the most important safety screen. Cells with elevated self-discharge rates may have internal micro-shorts that can lead to thermal events. The test requires a long rest period (7-14 days), which makes it the bottleneck in cell production. Track self-discharge data in TofuPilot to: - Set data-driven pass/fail thresholds based on production distributions - Identify cells that need extended screening - Correlate self-discharge with other parameters (formation data, impedance) ## Traceability from Cell to Pack When a pack fails in the field, trace back to the individual cells: 1. Look up the pack serial in TofuPilot 2. Find which cells were assembled into that pack 3. Pull each cell's grading data 4. Check if any cell was marginal at grading time This traceability is required by most automotive OEMs and is becoming standard across the battery industry. ### Multi-Site Test Data Management with TofuPilot URL: https://www.tofupilot.com/guides/multi-site-test-data-management-with-tofupilot Learn how to manage hardware test data across multiple manufacturing sites and test locations using TofuPilot's centralized platform. # Multi-Site Test Data Management with TofuPilot When you test hardware at multiple locations, data silos are inevitable. Factory A has its own test system. The contract manufacturer has another. Your lab uses something else entirely. TofuPilot centralizes test data from every site into one platform, so you can compare quality across locations. ## The Multi-Site Problem Companies scale production by adding sites: an in-house prototype lab, a contract manufacturer for volume production, a second CM for geographic diversification, field test stations at customer sites. Each site typically has: - Different test equipment and fixtures - Different data storage (local databases, CSV files, proprietary systems) - Different reporting formats - Different levels of data granularity The result: you can't answer basic questions like "Is the CM's yield the same as our in-house line?" without weeks of manual data gathering and normalization. ## Centralizing Test Data ### Step 1: Standardize Procedure IDs Use the same procedure ID at every site for the same test. This is the key to cross-site comparison. ```python filename="site_a_test.py" from tofupilot import TofuPilotClient client = TofuPilotClient() # Same procedure ID used at every site client.create_run( procedure_id="FINAL-FUNCTIONAL-V3", unit_under_test={ "serial_number": "UNIT-A-0042", "part_number": "PROD-100-R4", }, run_passed=True, steps=[{ "name": "Power Rail Check", "step_type": "measurement", "status": True, "measurements": [{ "name": "vcc_3v3", "value": 3.31, "unit": "V", "limit_low": 3.25, "limit_high": 3.35, }], }], ) ``` ### Step 2: Tag Runs with Site Information Use station metadata or sub-unit tracking to identify which site produced each run. ```python filename="site_b_test.py" from tofupilot import TofuPilotClient client = TofuPilotClient() client.create_run( procedure_id="FINAL-FUNCTIONAL-V3", unit_under_test={ "serial_number": "UNIT-B-1087", "part_number": "PROD-100-R4", }, run_passed=True, steps=[{ "name": "Power Rail Check", "step_type": "measurement", "status": True, "measurements": [{ "name": "vcc_3v3", "value": 3.29, "unit": "V", "limit_low": 3.25, "limit_high": 3.35, }], }], ) ``` ### Step 3: Compare Sites in the Dashboard With standardized procedure IDs, TofuPilot's station comparison view works across sites. Filter by station to see: | Metric | Site A (In-house) | Site B (CM) | |--------|-------------------|-------------| | FPY | 98.2% | 95.1% | | Avg vcc_3v3 | 3.31 V | 3.29 V | | Std dev vcc_3v3 | 0.012 V | 0.028 V | | Cycle time | 42 s | 55 s | This table tells a story: Site B has lower yield, lower average voltage, higher variance, and longer cycle time. The higher variance suggests a fixture or process control issue at the CM. ## Cross-Site Quality Governance ### Set Consistent Limits Use the same measurement limits at every site. TofuPilot enforces limits at the procedure level, so a unit that passes at Site A will also pass at Site B (assuming the measurements are accurate). ### Monitor Site-to-Site Differences Check these metrics regularly across sites: 1. **FPY gap**: If one site's yield is consistently lower, investigate 2. **Measurement distribution overlap**: Measurement histograms from different sites should overlap. If they don't, the sites are producing or testing differently 3. **Failure mode differences**: Different top failure modes at different sites suggest different process issues 4. **Cycle time variation**: Significantly different cycle times may indicate different test coverage or equipment issues ### Handle CM Data Integration Contract manufacturers often use their own test systems. Options for getting their data into TofuPilot: 1. **Direct integration**: CM installs TofuPilot client on their test stations. Cleanest solution, real-time data. 2. **Batch upload**: CM exports CSV/JSON files, you upload via API. Works when the CM can't install software on their stations. 3. **API relay**: CM pushes data to an intermediary that forwards to TofuPilot. Good for CMs with existing data systems. ```python filename="cm_batch_upload.py" import csv from tofupilot import TofuPilotClient client = TofuPilotClient() # Upload CM data from their export file with open("cm_test_results.csv") as f: reader = csv.DictReader(f) for row in reader: client.create_run( procedure_id="FINAL-FUNCTIONAL-V3", unit_under_test={"serial_number": row["serial"]}, run_passed=row["result"] == "PASS", steps=[{ "name": "Power Rail Check", "step_type": "measurement", "status": float(row["vcc_3v3"]) >= 3.25, "measurements": [{ "name": "vcc_3v3", "value": float(row["vcc_3v3"]), "unit": "V", "limit_low": 3.25, "limit_high": 3.35, }], }], ) ``` ## Fleet Telemetry Across Sites For companies deploying products globally (robotics fleets, industrial equipment, medical devices), TofuPilot provides a single view of test data regardless of where the unit was tested. Search by serial number to see a unit's full history: manufactured and tested at Site A, field-tested at the customer's location, returned and retested at Site B. Every measurement, every result, every timestamp in one place. This is especially valuable for warranty analysis. When a unit comes back for repair, you can compare its original production test data with its current state to understand what degraded. ### Test Traceability for IATF 16949 with TofuPilot URL: https://www.tofupilot.com/guides/test-traceability-for-iatf-16949-with-tofupilot Map IATF 16949 requirements for production test records, MSA, and control plans to TofuPilot's traceability and analytics features. IATF 16949 requires automotive suppliers to maintain complete test records for every production unit, track measurement system capability, and follow documented control plans. TofuPilot gives you the infrastructure to capture, store, and analyze all of that without building custom tooling. ## What IATF 16949 Requires for Test Records The standard builds on ISO 9001 and adds automotive-specific clauses that directly affect production testing. **Clause 8.5.2 (Identification and Traceability)** requires you to identify each product throughout production and retain records that link a serial number to its full test history. Every DUT must be traceable from raw material through final test. **Clause 7.1.5 (Measurement System Analysis)** mandates that you study the variation in your measurement systems. You need to demonstrate that your gauges, fixtures, and test equipment produce repeatable, reproducible results. This means tracking Cpk and GR&R across your measurement data over time. **Clause 8.5.1.1 (Control Plans)** requires documented control plans that specify what you measure, how often, and what the acceptance criteria are. Your test infrastructure needs to enforce these limits consistently. ## How TofuPilot Maps to IATF Requirements | IATF 16949 Clause | Requirement | TofuPilot Feature | |---|---|---| | 8.5.2 | Serial number traceability | Unit tracking with full run history per serial number | | 7.1.5.1.1 | MSA / GR&R studies | Measurement history and Cpk charts per measurement name | | 8.5.1.1 | Control plan enforcement | Measurement limits stored with every run | | 8.6 | Release of products | Pass/fail verdict tied to limit compliance | | 8.7 | Nonconforming output | Failure Pareto and filtering by failed measurements | | 9.1.1 | Performance monitoring | FPY trends, station throughput, control charts | ## Structuring Tests for Automotive Traceability Your OpenHTF test should capture the serial number (DUT ID), station identity, operator, and every measurement with its acceptance limits. This gives TofuPilot everything it needs to build a complete traceability record. ```python filename="automotive_ecu_test.py" import openhtf as htf from openhtf.util import units from tofupilot.openhtf import TofuPilot @htf.measures( htf.Measurement("supply_voltage") .in_range(minimum=4.75, maximum=5.25) .with_units(units.VOLT), htf.Measurement("quiescent_current") .in_range(maximum=0.012) .with_units(units.AMPERE), htf.Measurement("output_signal_frequency") .in_range(minimum=999.5, maximum=1000.5) .with_units(units.HERTZ), ) def power_and_signal_check(test): test.measurements.supply_voltage = 5.02 test.measurements.quiescent_current = 0.0087 test.measurements.output_signal_frequency = 1000.1 @htf.measures( htf.Measurement("can_bus_response_time") .in_range(maximum=10.0), htf.Measurement("can_message_checksum_valid") .equals(True), ) def communication_check(test): test.measurements.can_bus_response_time = 4.3 test.measurements.can_message_checksum_valid = True def main(): test = htf.Test( power_and_signal_check, communication_check, ) with TofuPilot(test): test.execute(test_start=lambda: "ECU-SN-20260312-0047") if __name__ == "__main__": main() ``` Every run recorded in TofuPilot links the serial number, station, measurements, limits, and pass/fail verdict into one traceable record. ## Measurement System Analysis with TofuPilot IATF 16949 requires you to prove your measurement systems are capable. TofuPilot tracks every measurement value with its limits across all runs, so you can monitor Cpk directly from the dashboard. To support MSA, structure your tests so that each measurement name stays consistent across stations and over time. If you measure `supply_voltage` on Station A and Station B, use the same measurement name on both. TofuPilot's measurement analytics will then show you the distribution, Cpk, and control chart for that measurement across your entire fleet. You don't need to write Python scripts to compute Cpk or build histograms. TofuPilot's dashboard shows process capability metrics, measurement histograms, and control charts for every measurement name you record. ## Control Plan Enforcement Your control plan defines what gets measured and the acceptance criteria. In OpenHTF, you encode these as measurement limits using `.in_range()`. TofuPilot stores both the measured value and the limits with every run. This means an auditor can verify that every unit was tested against the documented limits. If limits change between production batches, TofuPilot's run history shows exactly which limits applied to which units. ## Nonconforming Output and Failure Analysis When a unit fails, IATF 16949 requires you to document what went wrong and prevent recurrence. TofuPilot's failure Pareto chart shows which measurements fail most often across your production. Filter by station, time range, or part number to isolate systemic issues. Every failed run retains the exact measured values alongside the limits that were violated, giving you the raw data for 8D reports and corrective action records. ## Audit Readiness For IATF audits, you need to demonstrate that your test system captures complete records and that you monitor process capability. With TofuPilot, you can pull up any unit by serial number and show its full test history, measurements, limits, and verdict. The dashboard's FPY trends and Cpk charts provide the ongoing monitoring evidence auditors expect. ### How to Build KPI Dashboards with TofuPilot URL: https://www.tofupilot.com/guides/how-to-build-kpi-dashboards-with-tofupilot Learn how to structure your OpenHTF tests so TofuPilot automatically generates manufacturing KPI dashboards with FPY, throughput, Cpk, and failure analysis. TofuPilot generates manufacturing KPI dashboards from your test data automatically. You don't need to build reports in Python or connect BI tools for standard metrics. Write well-structured tests, and the dashboards populate themselves. ## Key Manufacturing KPIs These are the metrics that matter on the production floor. **First Pass Yield (FPY)** is the percentage of units that pass all tests on the first attempt. It's the single best indicator of process health. TofuPilot calculates FPY per procedure, per station, and across your entire line. **Throughput** measures units tested per hour or per shift. It tells you whether your line is hitting capacity targets. Bottlenecks show up as throughput drops on specific stations. **Cpk** (Process Capability Index) quantifies how centered your measurements are within spec limits. A Cpk above 1.33 means your process has comfortable margin. Below 1.0 means you're producing out-of-spec parts. **Cycle time** is how long each test takes. It's the denominator in throughput and the first thing to optimize when you need more capacity. **Failure rate by step** breaks down where failures occur. A Pareto chart of failure modes tells you exactly where to invest engineering effort. ## Structuring Tests for Complete KPIs Every KPI above comes from data your tests already produce. The key is structuring tests so TofuPilot can extract all of it cleanly. ```python filename="kpi_ready_test.py" import openhtf as htf from openhtf.util import units from tofupilot.openhtf import TofuPilot @htf.measures( htf.Measurement("boot_time") .in_range(maximum=3.0), htf.Measurement("firmware_version") .with_allowed_values("2.4.1", "2.4.2"), ) def firmware_validation(test): """Validate firmware boots correctly.""" test.measurements.boot_time = 1.8 test.measurements.firmware_version = "2.4.1" @htf.measures( htf.Measurement("battery_voltage") .with_units(units.VOLT) .in_range(minimum=3.6, maximum=4.2), htf.Measurement("charge_current") .in_range(minimum=0.450, maximum=0.550) .with_units(units.AMPERE), htf.Measurement("discharge_capacity") .in_range(minimum=2800), ) def battery_test(test): """Test battery charging and capacity.""" test.measurements.battery_voltage = 3.95 test.measurements.charge_current = 0.502 test.measurements.discharge_capacity = 3050 @htf.measures( htf.Measurement("touch_sensitivity_pct") .in_range(minimum=85.0), htf.Measurement("display_brightness") .in_range(minimum=400, maximum=600), htf.Measurement("pixel_defect_count") .in_range(maximum=0), ) def display_and_touch_test(test): """Verify display and touchscreen performance.""" test.measurements.touch_sensitivity_pct = 94.2 test.measurements.display_brightness = 520 test.measurements.pixel_defect_count = 0 @htf.measures( htf.Measurement("speaker_thd_pct") .in_range(maximum=1.0), htf.Measurement("microphone_snr") .in_range(minimum=60.0), ) def audio_test(test): """Test speaker and microphone quality.""" test.measurements.speaker_thd_pct = 0.4 test.measurements.microphone_snr = 68.5 def main(): test = htf.Test( firmware_validation, battery_test, display_and_touch_test, audio_test, ) with TofuPilot(test): test.execute(test_start=lambda: "DEVICE-007") if __name__ == "__main__": main() ``` This test gives TofuPilot everything it needs. Numeric measurements with limits feed Cpk calculations. Phase structure feeds step-level failure analysis. Timestamps feed throughput and cycle time. Pass/fail outcomes feed FPY. ## What Makes Tests KPI-Ready Three things make the difference between tests that produce useful dashboards and tests that don't. **Numeric measurements with limits.** Every measurement that has `in_range()` limits gets Cpk tracking, histograms, and control charts. Boolean pass/fail checks are fine for go/no-go tests, but they don't feed the analytics. **Consistent phase names.** If you rename a phase, TofuPilot treats it as a new step. Keep names stable so historical comparisons work. Use descriptive names that make sense in a dashboard context. **Unique serial numbers.** The DUT ID connects a unit's full test history across stations. Use real serial numbers, not placeholder values. This enables traceability and retest tracking. ## Viewing Dashboards in TofuPilot TofuPilot's dashboard shows your KPIs in real time as test results upload. You'll find: - FPY trends over time, filterable by procedure, station, or time range - Throughput charts showing units per hour by station - Measurement histograms and Cpk values for every numeric measurement - Failure Pareto charts ranking failure modes by frequency - Yield gauges for at-a-glance status Custom dashboards let you combine widgets for specific views. Build a line manager dashboard with FPY and throughput, or an engineering dashboard focused on Cpk and failure analysis. ## Keeping Dashboards Accurate Dashboards are only as good as the data behind them. Run every unit through the test, including units you already know will fail. Skipping known-bad units inflates your FPY. Use separate procedures for retest vs. first test. This keeps FPY calculations clean while still tracking retest outcomes. TofuPilot handles both and shows them independently. ### How to Improve First Pass Yield with TofuPilot URL: https://www.tofupilot.com/guides/how-to-improve-first-pass-yield-in-electronics-manufacturing-with-tofupilot Actionable strategies to improve first pass yield (FPY) in electronics manufacturing, with root cause analysis, Pareto charts, limit refinement, and TofuPilot. Low first pass yield costs more than scrap. It means retest cycles, debug time, and late shipments. This guide covers practical strategies to find what's failing, why it's failing, and how to fix it using production data from TofuPilot. ## Prerequisites - A TofuPilot account with test runs already flowing in - OpenHTF test scripts with measurements and limits defined - At least a few hundred runs for meaningful analysis ## Step 1: Understand Where You're Losing Yield Before fixing anything, you need to know what's actually failing. Most teams have a gut feeling about their worst tests, but the data usually tells a different story. TofuPilot's Analytics tab shows your FPY over time, broken down by procedure. Start there. Look for: - **Procedures with FPY below 95%**: these are your top targets - **FPY drops that correlate with a date**: usually a process change, new component lot, or test script update - **Stations with lower FPY than others running the same procedure**: points to fixturing or calibration issues The Pareto chart on the Analytics tab ranks failure modes by frequency. This is where you focus. Fixing the top two or three failure modes typically recovers most of your lost yield. ## Step 2: Run Root Cause Analysis on Top Failures Once you've identified your worst-performing measurements, dig into the data. There are three common root cause categories. ### Test limit issues Limits that are too tight cause false failures. Limits that are too loose let bad units through. Both hurt yield, just in different ways. TofuPilot's control charts show measurement trends with 3-sigma limits automatically. Use the Cpk view to identify measurements with poor process capability. A Cpk below 1.0 means your process spread is wider than your spec limits, and you'll keep failing units until you fix either the process or the limits. | Cpk Range | Interpretation | Action | |-----------|---------------|--------| | < 1.0 | Process not capable | Widen limits or improve process | | 1.0 - 1.33 | Barely capable | Monitor closely, plan improvement | | 1.33 - 1.67 | Capable | Acceptable for most production | | > 1.67 | Highly capable | Consider tightening limits | ### Process issues If a measurement's distribution shifts over time or between stations, that's a process issue, not a limit issue. Common causes: - Component lot variation (especially passives and connectors) - Fixture wear or contamination - Environmental changes (temperature, humidity in the test area) - Operator-dependent steps with inconsistent execution ### Test script issues Sometimes the test itself is the problem. Flaky measurements, missing settling time, or incorrect instrument configuration can all show up as yield loss. Look for measurements with bimodal distributions or high variance that doesn't correlate with the DUT. ## Step 3: Refine Your Measurement Limits Good limits come from a combination of the component datasheet, your design margins, and actual production data. Here's the workflow. ### Start with datasheet limits Your initial limits should reflect the design intent. Pull minimum and maximum values from the relevant datasheets and your design specs. ```python filename="tests/power_rail_test.py" import openhtf as htf from openhtf.util import units @htf.measures( htf.Measurement("vdd_3v3") .with_units(units.VOLT) .in_range(minimum=3.135, maximum=3.465) # 3.3V +/- 5% from regulator datasheet .doc("Main 3.3V rail, measured at C12"), htf.Measurement("vdd_1v8") .with_units(units.VOLT) .in_range(minimum=1.746, maximum=1.854) # 1.8V +/- 3% from LDO datasheet .doc("Core 1.8V rail, measured at C45"), ) def measure_power_rails(test): # Instrument reads go here test.measurements.vdd_3v3 = read_voltage("3V3_TP") test.measurements.vdd_1v8 = read_voltage("1V8_TP") ``` ### Tighten with production data After collecting a few hundred units, check the actual distribution in TofuPilot's control charts. If your measurements cluster tightly around the nominal with Cpk > 1.67, you've got room to tighten. Tighter limits catch marginal units before they become field returns. ### Add marginal limits for early warning OpenHTF supports marginal limits that flag units as "marginal pass" without outright failing them. This gives you early warning when a measurement is drifting toward spec limits. ```python filename="tests/power_rail_test.py" import openhtf as htf from openhtf.util import units @htf.measures( htf.Measurement("vdd_3v3") .with_units(units.VOLT) .in_range( minimum=3.135, maximum=3.465, marginal_minimum=3.168, marginal_maximum=3.432, # Inner 2% band ) .doc("Main 3.3V rail with marginal detection"), ) def measure_power_rails(test): test.measurements.vdd_3v3 = read_voltage("3V3_TP") ``` Marginal units pass but get flagged in TofuPilot. When marginal rates climb, you know a process shift is underway before it starts causing hard failures. ## Step 4: Improve Test Structure for Reliable Results Flaky tests kill yield numbers. A few structural practices make a big difference. ### Separate measurements cleanly Each measurement should test one thing. If a single phase measures five different voltages, a failure in any one of them makes it harder to identify the root cause. ```python filename="tests/board_test.py" import openhtf as htf from openhtf.util import units import time # Good: one measurement per phase, clear naming @htf.measures( htf.Measurement("supply_current_idle") .with_units(units.AMPERE) .in_range(minimum=0.010, maximum=0.025) .doc("Board idle current draw at 3.3V input"), ) def measure_idle_current(test): set_load("idle") time.sleep(0.5) # Settling time matters test.measurements.supply_current_idle = read_current("ISENSE") @htf.measures( htf.Measurement("supply_current_active") .with_units(units.AMPERE) .in_range(minimum=0.080, maximum=0.150) .doc("Board current draw during active processing"), ) def measure_active_current(test): set_load("active") time.sleep(1.0) # Active mode needs longer settling test.measurements.supply_current_active = read_current("ISENSE") ``` ### Add settling time and retries for noisy measurements If a measurement is inherently noisy (RF power, current draw during transitions), average multiple readings or add a retry with a short delay. Don't just hope for the best. ```python filename="tests/rf_test.py" import openhtf as htf import time @htf.measures( htf.Measurement("tx_power_dbm") .in_range(minimum=18.0, maximum=22.0) .doc("Transmit power at 2.4 GHz, averaged over 5 readings"), ) def measure_tx_power(test): readings = [] for _ in range(5): readings.append(read_rf_power("TX_OUT")) time.sleep(0.1) test.measurements.tx_power_dbm = sum(readings) / len(readings) ``` ### Use descriptive measurement names Names like `m1`, `test_3`, or `voltage` make it impossible to do root cause analysis at scale. Use names that describe what's being measured and where. | Bad Name | Good Name | |----------|-----------| | `voltage_1` | `vdd_3v3_at_c12` | | `test_pass` | `wifi_association_2g4` | | `current` | `supply_current_idle` | | `temp` | `pcb_temp_post_burn_in` | ## Step 5: Build a Continuous Improvement Workflow Improving FPY isn't a one-time project. It's a weekly practice. ### Weekly review process 1. **Check FPY trend** in TofuPilot's Analytics tab. Is it improving, flat, or declining? 2. **Review the Pareto chart** for top failure modes. Have the top failures changed since last week? 3. **Inspect control charts** for any measurements showing drift or increased variance 4. **Update limits** if production data shows Cpk values that justify tightening or loosening 5. **Track marginal rates** as a leading indicator of future yield loss ### When to act vs. when to monitor | Signal | Action | |--------|--------| | FPY drops > 2% in a week | Investigate immediately | | New failure mode enters top 3 | Root cause within 48 hours | | Cpk drops below 1.33 | Plan process improvement | | Marginal rate increases > 50% | Investigate component lots or fixture | | FPY stable above target | Monitor weekly, no action needed | ## Summary Improving FPY follows a repeatable pattern: identify the top failures with Pareto analysis, run root cause analysis using TofuPilot's control charts and Cpk data, refine limits based on production distributions, and monitor weekly for regressions. The teams that sustain high yield are the ones that treat this as a continuous loop, not a one-time fix. ### Data Sovereignty for Manufacturing Test Data URL: https://www.tofupilot.com/guides/data-sovereignty-for-manufacturing-test-data Learn why test data jurisdiction matters for manufacturing companies and how to protect process parameters and yield data from foreign surveillance laws. # Data Sovereignty for Manufacturing Test Data Your test results, process parameters, and yield data are trade secrets. Where that data is hosted, and which government can legally compel access to it, determines your actual exposure. This guide breaks down the laws, the risks, and the practical options. ## What Manufacturing Test Data Contains Test data isn't just pass/fail flags. A typical test database holds: | Data Type | Examples | Why It's Sensitive | |-----------|----------|--------------------| | Process parameters | Voltage thresholds, torque values, calibration offsets | Reveals manufacturing know-how | | Yield and quality metrics | FPY, Cpk, retest rates, failure Pareto | Exposes production maturity | | Serial genealogy | Serial numbers, sub-assembly tracking, revision history | Full product traceability | | Test sequences | Phase ordering, measurement limits, pass criteria | Core test IP | | Station metadata | Station IDs, operator logs, throughput data | Factory operational intelligence | For companies in defense, medtech, automotive, or aerospace, this data is regulated. For everyone else, it's still competitive intelligence you don't want a foreign government browsing. ## How US Surveillance Law Applies to SaaS Three US laws matter for non-US companies using US-based SaaS: | Law | What It Covers | Who It Compels | Key Detail | |-----|---------------|----------------|------------| | CLOUD Act (2018) | All data held by US providers, regardless of storage location | Any US company or company with sufficient US nexus | A US warrant reaches data in EU data centers if the provider is American | | FISA Section 702 | Communications and data of non-US persons outside the US | US electronic communication service providers | No individual warrant required for non-US targets. Reauthorized April 2024 | | National Security Letters | Subscriber metadata, transaction records | US companies | Issued by the FBI without a judge. Comes with a gag order by default | The practical risk of your test data being targeted by US intelligence is low. These tools focus on counter-terrorism, espionage, and cyber threats. But the legal possibility exists, and your customers' security teams will ask about it. ## Why EU Data Centers Don't Solve This A common misconception: "Our US vendor hosts in Frankfurt, so we're fine." The CLOUD Act explicitly states that US legal process applies to data controlled by US companies regardless of where it's stored. A US company running servers in `eu-west-1` is still a US company. The EU has tried to solve this with data transfer frameworks. All of them have been fragile: | Framework | Years Active | What Happened | |-----------|-------------|---------------| | Safe Harbor | 2000-2015 | Invalidated by EU Court of Justice (Schrems I) | | Privacy Shield | 2016-2020 | Invalidated by EU Court of Justice (Schrems II) | | Data Privacy Framework | 2023-present | Active, but built on a US executive order that can be revoked. Challenge expected | Relying on transfer frameworks means accepting the risk that your legal basis for data transfers could disappear overnight, as it did twice already. ## What Actually Protects Your Data Protection comes in layers. No single measure is absolute. | Layer | US SaaS on AWS | Non-US SaaS on US infra | Non-US SaaS, non-US infra | Self-Hosted | |-------|---------------|------------------------|---------------------------|-------------| | App provider compellable by US law | Yes | No | No | No | | Infra provider compellable by US law | Yes | Yes | No | No | | Encryption at rest | Varies | Yes | Yes | Yes | | Full jurisdiction control | No | No | Depends on provider | Yes | The key insight: using a non-US SaaS provider on US-owned infrastructure removes one attack vector (the app provider can't be compelled), but the infrastructure provider remains subject to US law. Only self-hosting or using non-US infrastructure removes both. For most manufacturing companies, the combination of a non-US app provider with encryption at rest provides a strong practical posture. The infrastructure provider holds encrypted data but has no context about what it contains. For regulated industries or classified environments, self-hosting is the only option that provides full sovereignty. ## How TofuPilot Handles Data Sovereignty TofuPilot SA is a Swiss-incorporated company with no US legal entity. US surveillance laws cannot compel TofuPilot to disclose customer data. **Cloud deployment:** - Database and file storage hosted in EU - Data encrypted at rest and in transit - TofuPilot operates under Swiss data protection law (nFADP), recognized as adequate by the EU - No dependency on the EU-US Data Privacy Framework **Self-hosted deployment:** - Single Docker image, runs on your infrastructure - Full air-gap support with no external dependencies - Zero data leaves your network - All features available, including analytics and traceability | Concern | Cloud | Self-Hosted | |---------|-------|-------------| | TofuPilot compellable by US law | No | No | | Infrastructure under US jurisdiction | Partially (US-owned infra providers) | No (your servers) | | GDPR compliance | Yes, by corporate structure | Yes, fully on-premise | | Air-gap support | No | Yes | For companies that need to answer "where is our test data and who can access it" in a vendor security review, TofuPilot provides a clear answer at both the application and infrastructure level. ### What Is SPC (Statistical Process Control) URL: https://www.tofupilot.com/guides/what-is-spc-statistical-process-control SPC uses production test data to separate normal variation from real problems. Learn the core tools, how they connect, and how TofuPilot automates SPC. Statistical Process Control (SPC) is a method for monitoring and controlling a manufacturing process using data. Instead of inspecting quality at the end, SPC tracks measurements continuously to catch problems while they're still fixable. The core idea: every process has variation, and SPC helps you tell the difference between normal noise and something that actually changed. ## Common Cause vs Special Cause Variation All manufacturing processes produce variation. SPC splits it into two types. **Common cause variation** is inherent to the process. Instrument noise, slight material differences, ambient temperature fluctuations. It's random, stable, and predictable within a range. You can't eliminate it without fundamentally changing the process. **Special cause variation** is something new. A calibration drifted, a component lot changed, a fixture pin bent. It's non-random and signals that the process changed. SPC tools are designed to detect this. The goal isn't zero variation. It's knowing which type you're looking at so you take the right action. Adjusting a process in response to common cause variation (overreacting to noise) actually makes things worse. ## The SPC Toolkit SPC has a small set of tools. Each answers a different question about your process. | Tool | Question It Answers | What to Look For | |------|-------------------|-----------------| | Control chart | Is my process stable over time? | Points outside control limits, patterns, trends | | Histogram | What does my distribution look like? | Shape, centering, spread relative to spec limits | | Capability indices (Cp, Cpk) | Can my process meet spec short-term? | Values above 1.33 | | Performance indices (Pp, Ppk) | Can my process meet spec long-term? | Values close to Cp/Cpk (stable process) | | Cpk trend | Is capability improving or degrading? | Sustained direction changes | These tools work together. A control chart tells you the process shifted. The histogram tells you how the distribution changed. Capability indices quantify whether you still meet spec. ## Control Charts A control chart plots measurement values over time against three lines: a center line (process mean), an upper control limit (UCL), and a lower control limit (LCL). Control limits are calculated from your data at ±3σ from the mean. Points inside the limits mean the process is behaving normally. Signals that something changed: - A single point beyond UCL or LCL - Seven consecutive points on one side of the center line - Six consecutive points trending in one direction - Alternating up-down patterns (possible measurement system issue) Control limits are not spec limits. Spec limits (USL/LSL) come from your product requirements. Control limits come from your process data. A process can be in control but out of spec, or in spec but out of control. SPC catches the second case, which spec-only monitoring misses. ## Histograms A histogram groups measurement values into bins and shows how many fall in each. Overlaid with a normal distribution curve and spec limits, it reveals the shape and position of your process. | Shape | What It Means | |-------|-------------| | Normal (bell curve) | Process is predictable. Variation is random. | | Skewed | One-sided drift. Check for a physical constraint or asymmetric tolerance. | | Bimodal (two peaks) | Two populations mixed. Possible causes: two stations, two component lots, two operators. | | Truncated (cut off at one end) | Parts at the limit are being screened out or the spec is acting as a wall. | | Flat (uniform) | Process isn't controlled. Something is varying widely. | TofuPilot renders histograms alongside control charts for each measurement. The normal curve overlay and spec limit lines appear automatically when limits are defined. ## Capability Indices Capability indices reduce your process to a single number that says "can we meet spec?" **Short-term (Cp family):** Uses sample σ (n-1 divisor). - **Cp** = (USL - LSL) / 6σ. How much of the spec window the process uses. - **Cpk** = min(Cpu, Cpl). How well the process fits within spec, accounting for centering. - **Cpu** = (USL - X̄) / 3σ. Distance from mean to upper limit. - **Cpl** = (X̄ - LSL) / 3σ. Distance from mean to lower limit. **Long-term (Pp family):** Uses overall σo (n divisor). Same formulas, different σ. - **Pp**, **Ppk**, **Ppu**, **Ppl** When Cpk and Ppk are close, the process is stable. When Ppk is lower, there's hidden variation between batches. TofuPilot shows both families side by side in the Capability tab. ## Setting Up SPC in TofuPilot SPC starts with well-defined measurements and limits. Write your tests with explicit spec limits so TofuPilot has the data it needs. ```python filename="spc_ready_test.py" import openhtf as htf from openhtf.util import units from tofupilot.openhtf import TofuPilot @htf.measures( htf.Measurement("supply_voltage") .in_range(minimum=4.75, maximum=5.25) .with_units(units.VOLT), htf.Measurement("current_draw") .in_range(minimum=0.090, maximum=0.110) .with_units(units.AMPERE), htf.Measurement("frequency") .in_range(minimum=999.5, maximum=1000.5) .with_units(units.HERTZ), ) def functional_check(test): test.measurements.supply_voltage = 5.02 test.measurements.current_draw = 0.098 test.measurements.frequency = 1000.1 def main(): test = htf.Test(functional_check) with TofuPilot(test): test.execute(test_start=lambda: "PCB-0001") if __name__ == "__main__": main() ``` Once 30+ runs accumulate, open the Process Control page for your procedure. Select a measurement to see its control chart, histogram, and capability indices. The measurement overview ranks all measurements by failure count, fail rate, or Cpk so you can focus on the worst performers first. ## When to Act | Signal | Meaning | Response | |--------|---------|----------| | Point beyond UCL/LCL | Special cause event | Investigate immediately. Check last lot change, calibration, operator. | | 7 points same side of mean | Process has shifted | Check calibration, component lot, environmental change. | | Cpk dropping over days | Capability degrading | Investigate drift source before it causes failures. | | Cpk < 1.0 | Not capable | Reduce σ (process improvement) or widen spec (if justified). | | Cpk >> Ppk | Short-term fine, long-term not | Between-batch variation. Investigate lot, operator, or environmental factors. | | Cp >> Cpk | Capable but off-center | Re-center the process. Calibration or recipe adjustment. | | Histogram bimodal | Two populations | Separate by station, operator, or lot. Fix the source. | ## What SPC Won't Tell You SPC detects that something changed. It doesn't tell you why. When a control chart signals a special cause, you still need to investigate: check the operator log, review the component lot, inspect the fixture, look at environmental data. SPC narrows the search. Domain knowledge finishes the job. ### What Is Cpk (Process Capability Index) URL: https://www.tofupilot.com/guides/what-is-cpk-process-capability-index Cpk measures whether your process produces units within spec limits, accounting for centering. Learn the formula, thresholds, and how TofuPilot tracks it. Cpk is the process capability index. It tells you whether your manufacturing process can consistently produce units within specification limits, accounting for how centered the process mean is between those limits. A Cpk of 1.33 or higher is the standard minimum for production processes. ## The Formula Cpk takes the worse of two ratios: the distance from the process mean to each spec limit, divided by 3 times the process standard deviation. **Cpk = min((USL - X̄) / 3σ, (X̄ - LSL) / 3σ)** Where: - USL is the upper specification limit - LSL is the lower specification limit - X̄ is the process mean - σ is the sample standard deviation (n-1 divisor) The "min" is what makes Cpk useful. It always reports the worst side. If your process drifts toward one limit, Cpk drops even if the other side has plenty of room. ## What Cpk Values Mean | Cpk | Interpretation | Approx. Defect Rate | |-----|---------------|---------------------| | < 0.67 | Poor. Significant portion of units out of spec. | > 45,500 ppm | | 0.67 | Process spread equals spec width on the worst side. | ~45,500 ppm | | 1.0 | Barely capable. The 3σ edge touches the spec limit. | ~2,700 ppm | | 1.33 | Standard minimum for production. 25% buffer for drift. | ~63 ppm | | 1.67 | Good. Required for safety-critical parameters. | ~0.6 ppm | | 2.0 | Excellent. Process uses half the spec window or less. | ~0.002 ppm | Most automotive (IATF 16949) and aerospace (AS9100) standards require Cpk >= 1.33 for production parameters. Safety-critical dimensions typically need 1.67. ## Cpk vs Cp Cp measures potential capability (how much of the spec window your variation uses) but ignores centering. Cpk accounts for centering. When Cp equals Cpk, the process is perfectly centered. When Cpk is much lower than Cp, the process mean has drifted off-center. Fix the centering (a calibration issue) and Cpk rises to match Cp. ## Cpk vs Ppk Cpk uses the sample standard deviation (n-1 divisor), which reflects short-term variation. Ppk uses the overall standard deviation (n divisor), which captures long-term variation across batches, shifts, and operators. If Ppk is much lower than Cpk, something is changing between production runs that your short-term samples don't capture. Investigate batch-to-batch variation, operator differences, or environmental shifts. ## How TofuPilot Calculates Cpk Define measurements with upper and lower limits in your test code. TofuPilot computes Cpk automatically from the accumulated production data. ```python filename="test_with_limits.py" import openhtf as htf from openhtf.util import units from tofupilot.openhtf import TofuPilot @htf.measures( htf.Measurement("output_voltage") .in_range(minimum=2.45, maximum=2.55) .with_units(units.VOLT), htf.Measurement("leakage_current") .in_range(maximum=0.001) .with_units(units.AMPERE), ) def validate_output(test): test.measurements.output_voltage = 2.501 test.measurements.leakage_current = 0.00042 def main(): test = htf.Test(validate_output) with TofuPilot(test): test.execute(test_start=lambda: "UNIT-0042") if __name__ == "__main__": main() ``` Open the Process Control page for your procedure, select a measurement, and switch to the Capability tab. TofuPilot shows Cpk alongside Cp, Cpl, and Cpu, with hover tooltips for each formula. The Cpk trend chart tracks daily Cpk over time so you can see whether capability is improving or degrading. Reference lines at 0.67, 1.0, and 1.33 make it easy to spot when the process crosses a threshold. ## Common Pitfalls **Not enough data.** Cpk from 10 units is unreliable. Wait for 30+ data points before making process decisions. **One-sided specs.** When you only set a maximum (like leakage current), TofuPilot computes a one-sided capability index (Cpu only). That's correct. You can't calculate a two-sided Cpk without both limits. **Mixing populations.** Calculating Cpk across data from two stations with different calibrations inflates σ and underestimates capability. Filter by station first. ### What Is Cp (Potential Process Capability) URL: https://www.tofupilot.com/guides/what-is-cp-potential-process-capability Cp measures how much of the spec window your process variation uses, ignoring centering. Learn the formula, how Cp relates to Cpk, and read it in TofuPilot. Cp is the potential process capability index. It compares the width of your specification window to the width of your process variation. Cp tells you the best Cpk you could achieve if your process were perfectly centered between the spec limits. ## The Formula **Cp = (USL - LSL) / 6σ** Where: - USL is the upper specification limit - LSL is the lower specification limit - σ is the within-subgroup (sample) standard deviation The numerator is the spec window. The denominator is 6σ, which covers 99.73% of a normal distribution. A Cp of 1.0 means your process variation exactly fills the spec window. Above 1.0, there's room to spare. Below 1.0, variation exceeds the spec range. ## What Cp Values Mean | Cp | Interpretation | |----|---------------| | < 1.0 | Process variation exceeds spec window. Even if perfectly centered, some units fall outside limits. | | 1.0 | Variation exactly matches spec width. No margin for drift. | | 1.33 | Process uses 75% of the spec window. Standard production target. | | 1.67 | Process uses 60% of the spec window. Safety-critical target. | | 2.0 | Process uses 50% of the spec window. | ## Why Cp Alone Isn't Enough Cp ignores where the process mean sits. A process with Cp = 1.5 could be perfectly centered (great) or shifted so far that units are failing on one side (bad). That's why Cp is always read alongside Cpk. | Scenario | Cp | Cpk | What It Tells You | |----------|-----|------|-------------------| | Centered, capable | 1.5 | 1.5 | Ideal state. | | Off-center, capable spread | 1.5 | 0.8 | Enough spread margin, but the mean drifted. Re-center it. | | Centered, too much variation | 0.8 | 0.8 | Variation is the problem. Reduce σ. | When Cp is good but Cpk is low, the fix is a centering adjustment (calibration, tool offset, recipe change). When Cp itself is low, you need to reduce fundamental process variation. ## Cp Requires Both Spec Limits Cp needs USL and LSL. For one-sided specs (like maximum leakage current or maximum response time), Cp can't be calculated. TofuPilot shows "--" for Cp when only one limit is defined, and reports Cpu or Cpl instead. ## Reading Cp in TofuPilot Open the Process Control page, select a numeric measurement, and switch to the Capability tab. Cp appears in the top KPI row alongside Cpk, Cpl, and Cpu. Click the Cp card to toggle its line on the daily trend chart. Hover over the Cp value to see the formula and a plain-language explanation. If Cp is significantly higher than Cpk, the tooltip confirms that re-centering the process would improve capability. ```python filename="two_sided_limits.py" import openhtf as htf from openhtf.util import units from tofupilot.openhtf import TofuPilot @htf.measures( htf.Measurement("resistance") .in_range(minimum=95, maximum=105) .with_units(units.OHM), ) def measure_resistance(test): test.measurements.resistance = 100.3 def main(): test = htf.Test(measure_resistance) with TofuPilot(test): test.execute(test_start=lambda: "UNIT-0101") if __name__ == "__main__": main() ``` Both `minimum` and `maximum` are set, so TofuPilot can compute Cp. If you only set one, you'll get a one-sided index instead. ### What Is Ppk (Process Performance Index) URL: https://www.tofupilot.com/guides/what-is-ppk-process-performance-index Ppk measures long-term process performance using overall standard deviation. Learn how it differs from Cpk and how TofuPilot tracks both. Ppk is the process performance index. It measures the same thing as Cpk (how well your process fits within spec limits) but uses the overall standard deviation instead of the sample standard deviation. This makes Ppk a long-term metric that captures all sources of variation: batch-to-batch, shift-to-shift, operator-to-operator, and environmental. ## The Formula **Ppk = min((USL - X̄) / 3σo, (X̄ - LSL) / 3σo)** Where: - USL is the upper specification limit - LSL is the lower specification limit - X̄ is the process mean - σo is the overall standard deviation (n divisor) The only difference from Cpk is the standard deviation. Cpk uses σ (sample, n-1 divisor). Ppk uses σo (overall, n divisor). ## Sample vs Overall Standard Deviation This distinction matters. The sample σ (n-1 divisor) provides an unbiased estimate of process variation and is used for short-term capability. The overall σo (n divisor) captures the total observed variation across the full dataset, including all between-run sources. | Statistic | Divisor | Captures | |-----------|---------|----------| | Sample σ | n-1 | Short-term variation: instrument noise, part-to-part within a batch | | Overall σo | n | All variation: batch shifts, operator differences, temperature changes, lot variation | Overall σo is always equal to or larger than sample σ. So Ppk is always equal to or lower than Cpk. ## What the Gap Between Cpk and Ppk Tells You When Cpk and Ppk are close, your process is consistent over time. The variation you see in a single batch is representative of the whole production run. When Cpk is significantly higher than Ppk, something is changing between batches. Your process looks capable in any given short window, but over days or weeks, extra variation appears. | Cpk | Ppk | Diagnosis | |-----|-----|-----------| | 1.5 | 1.4 | Process is stable. Short-term and long-term variation are similar. | | 1.5 | 0.9 | Hidden variation between batches. Investigate lot changes, operator differences, or environmental shifts. | | 0.8 | 0.8 | Process isn't capable even short-term. Fix fundamental variation first. | ## When to Use Ppk vs Cpk Use Cpk to assess inherent process capability under controlled conditions. Use Ppk to assess real-world performance over time. PPAP (Production Part Approval Process) in automotive typically requires Ppk >= 1.67 for initial process studies. Ongoing production monitoring uses Cpk >= 1.33. ## Reading Ppk in TofuPilot Open the Process Control page, select a numeric measurement, and switch to the Capability tab. The second KPI row shows the Pp family: Ppk, Pp, Ppl, and Ppu in teal. The first row shows the Cp family in purple. Compare the two rows at a glance. If the teal values are noticeably lower than the purple ones, you have between-batch variation to investigate. Click any index to toggle its line on the daily trend chart. Tracking Ppk alongside Cpk over time reveals when long-term variation starts growing. ## Common Pitfalls **Small datasets blur the distinction.** With fewer than 50 data points from a single batch, σ and σo converge. The Cpk/Ppk gap only becomes meaningful with data spanning multiple batches, shifts, or days. **Don't compare Ppk across different time windows without context.** Ppk over 7 days will differ from Ppk over 90 days because more sources of variation enter the calculation. ### What Is Pp (Overall Process Performance) URL: https://www.tofupilot.com/guides/what-is-pp-overall-process-performance Pp measures long-term process spread relative to spec width using overall standard deviation. Learn the formula, compare Pp to Cp, and track it in TofuPilot. Pp is the overall process performance index. It compares the specification window to the total process variation measured using the overall (population) standard deviation. Pp is to Ppk what Cp is to Cpk: it measures spread without accounting for centering. ## The Formula **Pp = (USL - LSL) / 6σo** Where: - USL is the upper specification limit - LSL is the lower specification limit - σo is the overall (population) standard deviation The structure is identical to Cp. The only difference is σo (overall) instead of σ (within-subgroup). This means Pp captures all variation: within-batch, between-batch, between-shift, and between-operator. ## Pp vs Cp Cp uses the within-subgroup σ and answers: "If we only look at short-term variation, does the spec window have room?" Pp uses the overall σo and answers: "Over the full production run, does the spec window have room?" | Index | Standard Deviation | Time Horizon | Question | |-------|-------------------|--------------|----------| | Cp | Within-subgroup σ | Short-term | Can this process fit within specs under controlled conditions? | | Pp | Overall σo | Long-term | Does this process fit within specs over real production? | When Pp is close to Cp, the process is stable over time. When Pp is noticeably lower than Cp, there's hidden variation between production runs that inflates the overall spread. ## What Pp Values Mean | Pp | Interpretation | |----|---------------| | < 1.0 | Long-term variation exceeds spec window. Process can't consistently meet specs over time. | | 1.0 | Long-term variation exactly fills the spec window. | | 1.33 | Process uses 75% of the spec window long-term. Standard target. | | > 1.67 | Process has significant long-term margin. | ## Reading Pp in TofuPilot Open the Process Control page, select a numeric measurement, and switch to the Capability tab. Pp appears in the second (teal) KPI row alongside Ppk, Ppl, and Ppu. The first (purple) row shows the Cp family. If Pp is lower than Cp, your process has more variation over time than within individual batches. That points to batch-level or environmental sources of variation worth investigating. Like all indices on the Capability tab, click Pp to toggle its trend line on the daily chart. Watching Pp over weeks helps you see whether long-term variation is growing or shrinking. ## Pp Requires Both Spec Limits Like Cp, Pp needs both USL and LSL. For one-sided specifications, TofuPilot computes Ppu (upper only) or Ppl (lower only) instead. ## When Pp Matters Most Pp is particularly useful during initial process validation (PPAP, process qualification). It tells you whether the process can sustain capability over a meaningful production window, not just during an optimized short run. Automotive PPAP studies typically require Pp >= 1.67 before granting production approval. ### What Is Cpl and Cpu (One-Sided Capability) URL: https://www.tofupilot.com/guides/what-is-cpl-and-cpu-one-sided-capability Cpl and Cpu measure process capability against individual spec limits. Learn the formulas, when to use one-sided indices, and how TofuPilot displays them. Cpl and Cpu are one-sided capability indices. Cpl measures how far the process mean is from the lower specification limit. Cpu measures how far it is from the upper specification limit. Together they form Cpk: Cpk = min(Cpl, Cpu). ## The Formulas **Cpu = (USL - X̄) / 3σ** **Cpl = (X̄ - LSL) / 3σ** Where: - USL is the upper specification limit - LSL is the lower specification limit - X̄ is the process mean - σ is the sample standard deviation (n-1 divisor) Each index expresses the distance from the process mean to one spec limit in units of 3σ. A value of 1.0 means the 3σ boundary exactly touches that limit. Above 1.0, there's margin. Below 1.0, some units are likely exceeding that limit. ## How Cpl and Cpu Build Cpk Cpk is simply the minimum of the two: **Cpk = min(Cpl, Cpu)**. This means Cpk always reflects the side that's closest to its spec limit. But looking at Cpl and Cpu separately tells you more than Cpk alone. | Cpl | Cpu | Cpk | What It Tells You | |-----|-----|-----|-------------------| | 1.8 | 1.2 | 1.2 | Process is closer to the upper limit. Mean is shifted high. | | 0.9 | 1.6 | 0.9 | Process is closer to the lower limit. Mean is shifted low. | | 1.4 | 1.4 | 1.4 | Process is centered. Both sides have equal margin. | When Cpk is low, Cpl and Cpu tell you which direction to adjust. If Cpu is the bottleneck, the process mean needs to shift down. If Cpl is the bottleneck, it needs to shift up. ## One-Sided Specifications Many measurements only have one spec limit. Maximum leakage current, maximum response time, minimum signal strength. For these, only one index applies. ```python filename="one_sided_specs.py" import openhtf as htf from openhtf.util import units from tofupilot.openhtf import TofuPilot @htf.measures( # Upper limit only: TofuPilot computes Cpu htf.Measurement("leakage_current") .in_range(maximum=0.001) .with_units(units.AMPERE), # Lower limit only: TofuPilot computes Cpl htf.Measurement("signal_strength") .in_range(minimum=-80), # Both limits: TofuPilot computes Cpl, Cpu, and Cpk htf.Measurement("output_voltage") .in_range(minimum=4.75, maximum=5.25) .with_units(units.VOLT), ) def measure_board(test): test.measurements.leakage_current = 0.00034 test.measurements.signal_strength = -62 test.measurements.output_voltage = 5.01 def main(): test = htf.Test(measure_board) with TofuPilot(test): test.execute(test_start=lambda: "PCB-0042") if __name__ == "__main__": main() ``` TofuPilot handles this automatically. When only `maximum` is set, it shows Cpu and displays "--" for Cpl. When only `minimum` is set, it shows Cpl and displays "--" for Cpu. ## Reading Cpl and Cpu in TofuPilot Open the Process Control page, select a numeric measurement, and switch to the Capability tab. The first KPI row shows Cpk, Cp, Cpl, and Cpu left to right. Hover over Cpl or Cpu to see its formula. Click either to toggle its line on the daily trend chart. Tracking Cpl and Cpu separately over time reveals directional drift. If Cpu is falling while Cpl is rising, the process mean is shifting upward toward USL. ## Ppl and Ppu The long-term equivalents are Ppl and Ppu. Same formulas but with overall σo (n divisor) instead of sample σ (n-1 divisor). They appear in the second (teal) KPI row and help you see whether one-sided drift is a short-term event or a persistent long-term trend. ### What Is Ppl and Ppu (One-Sided Performance) URL: https://www.tofupilot.com/guides/what-is-ppl-and-ppu-one-sided-performance Ppl and Ppu measure long-term process performance against individual spec limits. Learn the formulas, how they relate to Cpl/Cpu, and track them in TofuPilot. Ppl and Ppu are one-sided performance indices. They measure the distance from the process mean to each spec limit using the overall standard deviation. Ppl covers the lower limit, Ppu covers the upper. Together they form Ppk: Ppk = min(Ppl, Ppu). ## The Formulas **Ppu = (USL - X̄) / 3σo** **Ppl = (X̄ - LSL) / 3σo** Where: - USL is the upper specification limit - LSL is the lower specification limit - X̄ is the process mean - σo is the overall standard deviation (n divisor) These are identical to Cpu and Cpl, except they use σo (overall, n divisor) instead of σ (sample, n-1 divisor). This makes Ppl and Ppu long-term metrics that include all sources of variation. ## Ppl/Ppu vs Cpl/Cpu | Index Pair | Standard Deviation | What It Captures | |-----------|-------------------|-----------------| | Cpl / Cpu | Sample σ (n-1 divisor) | Short-term margin to each spec limit | | Ppl / Ppu | Overall σo (n divisor) | Long-term margin to each spec limit | If Cpu is 1.5 but Ppu is 0.9, the process has enough short-term margin on the upper side but loses it over time. Something between batches is pushing the distribution toward USL. Conversely, if Cpl and Ppl are similar, the lower-side margin is stable. ## Directional Drift Detection The real value of one-sided indices (both Cp and Pp families) is diagnosing which direction your process is drifting. | Signal | Meaning | Action | |--------|---------|--------| | Ppu dropping, Ppl stable | Process mean drifting toward USL over time | Investigate upward drift: calibration, temperature, component lots | | Ppl dropping, Ppu stable | Process mean drifting toward LSL over time | Investigate downward drift | | Both dropping | Overall variation increasing | Reduce σ, don't adjust mean | | Ppu < Cpu | Long-term upper margin worse than short-term | Between-batch variation biased upward | ## Reading Ppl and Ppu in TofuPilot Open the Process Control page, select a numeric measurement, and switch to the Capability tab. The second (teal) KPI row shows Ppk, Pp, Ppl, and Ppu. Compare the teal row to the purple row above it. If Ppu is significantly lower than Cpu, or Ppl is significantly lower than Cpl, the gap tells you where long-term variation is hurting you most. Click any index to toggle it on the daily trend chart. Plotting Ppl and Ppu together over weeks reveals whether the process is drifting toward one spec limit over time. ## When One-Sided Specs Apply For measurements with only an upper limit (leakage, noise, response time), TofuPilot shows Ppu and "--" for Ppl. For measurements with only a lower limit (signal strength, gain), TofuPilot shows Ppl and "--" for Ppu. This is the correct behavior. There's no opposing limit to measure against. ### Cpk vs Ppk: What's the Difference URL: https://www.tofupilot.com/guides/cpk-vs-ppk-whats-the-difference Cpk and Ppk both measure process capability, but use different standard deviations. Learn when to use each, what the gap reveals, and compare them in TofuPilot. Cpk and Ppk answer the same question (can my process meet spec?) but over different time horizons. Cpk uses the sample standard deviation and reflects short-term capability. Ppk uses the overall standard deviation and reflects long-term performance. The gap between them tells you whether your process is stable over time. ## The Formulas Side by Side | Index | Formula | Standard Deviation | |-------|---------|-------------------| | Cpk | min((USL - X̄) / 3σ, (X̄ - LSL) / 3σ) | Sample σ (n-1 divisor) | | Ppk | min((USL - X̄) / 3σo, (X̄ - LSL) / 3σo) | Overall σo (n divisor) | The formulas are identical except for σ. That one difference changes the meaning. ## Sample vs Overall Standard Deviation The sample σ (n-1 divisor) provides an unbiased estimate of process variation. For individual measurements (one value per unit, which is the typical case in electronics testing), this captures short-term variation under controlled conditions. The overall σo (n divisor) captures the total observed variation across the full dataset, including all between-run sources: batch-to-batch shifts, operator differences, temperature changes, component lot variation, fixture wear. Overall σo is always >= sample σ. So Ppk is always <= Cpk. When they're equal, the process is perfectly stable over time. When Ppk is lower, there's hidden variation between production runs. ## What the Gap Tells You | Cpk | Ppk | Gap | Diagnosis | |-----|-----|-----|-----------| | 1.5 | 1.45 | Small | Process is stable. Short-term and long-term variation are similar. No action needed. | | 1.5 | 1.0 | Large | Process looks capable in any given batch but degrades over time. Investigate what changes between batches. | | 1.5 | 0.7 | Very large | Significant between-batch variation. Common causes: component lot changes, operator training gaps, environmental shifts. | | 0.8 | 0.8 | None | Process isn't capable even short-term. Fix fundamental variation before worrying about batch-to-batch stability. | A large Cpk-Ppk gap is a signal to investigate sources of variation that don't show up within a single batch. Common culprits: - **Component lot variation.** A new reel of resistors shifts the circuit's behavior slightly. Each lot is in spec, but the distribution center moves. - **Operator differences.** Different fixture handling, different placement pressure, different ambient temperature from body heat near the DUT. - **Environmental changes.** Morning vs afternoon temperature, seasonal humidity shifts, facility voltage fluctuations. - **Fixture wear.** Pogo pin spring force degrading over thousands of insertions. Still passing, but adding variation. ## When to Use Each | Context | Use | Why | |---------|-----|-----| | Ongoing production monitoring | Cpk | You want to know if the process is inherently capable under controlled conditions. | | PPAP / initial process study | Ppk | Automotive standards require Ppk >= 1.67 to prove real-world performance. | | Process qualification | Both | Compare them. A large gap means the process isn't ready for unsupervised production. | | Root cause analysis | Both | If Cpk is fine but Ppk is low, the problem is between batches, not within them. | ## Industry Requirements | Standard | Cpk Requirement | Ppk Requirement | |----------|----------------|----------------| | IATF 16949 (automotive) | >= 1.33 ongoing | >= 1.67 initial study | | AS9100 (aerospace) | >= 1.33 production | >= 1.33 qualification | | ISO 13485 (medical) | Per product risk | Per product risk | | General electronics | >= 1.33 typical | >= 1.33 typical | ## Comparing Cpk and Ppk in TofuPilot Open the Process Control page, select a numeric measurement, and switch to the Capability tab. The purple row shows Cpk (with Cp, Cpl, Cpu). The teal row shows Ppk (with Pp, Ppl, Ppu). You can compare both families at a glance. Click any index to toggle it on the daily trend chart. Plotting Cpk and Ppk together over time is the clearest way to see whether the gap is growing, shrinking, or stable. If you notice Ppk trending downward while Cpk stays flat, something external to the process is introducing variation. Filter by station, operator, or batch in the sidebar to isolate the source. ## Quick Decision Guide **Cpk high, Ppk high:** Process is capable and stable. Monitor and maintain. **Cpk high, Ppk low:** Process is capable within batches but unstable over time. Investigate between-batch variation. **Cpk low, Ppk low:** Process isn't capable. Reduce variation first (σ), then worry about stability. **Cpk low, Ppk high:** Unusual. Possible data issue or very small dataset. Verify your data filters. ### What Is a Histogram in Manufacturing Test URL: https://www.tofupilot.com/guides/what-is-a-histogram-in-manufacturing-test A histogram shows the distribution shape of your measurement data. Learn how to read histogram patterns, spot problems, and use them for SPC in TofuPilot. A histogram groups your measurement values into bins and shows how many fall in each. It's the simplest way to see whether your process is centered, how much variation it has, and whether the distribution shape is normal or hiding a problem. In SPC, histograms complement control charts: the chart shows trends over time, the histogram shows the overall distribution shape. ## How to Read a Histogram The x-axis shows measurement values. The y-axis shows how many data points fall in each bin. A well-behaved process produces a bell curve (normal distribution) centered between the spec limits. Three things to look at: **Center.** Is the peak of the distribution aligned with the target value, or shifted toward one spec limit? A shifted histogram means the process mean has drifted. **Spread.** How wide is the distribution relative to the spec limits? A narrow distribution within wide spec limits means high capability (high Cp). A distribution that fills or exceeds the spec window means low capability. **Shape.** A symmetric bell curve is normal. Anything else is a signal worth investigating. ## Distribution Shapes and What They Mean | Shape | What It Looks Like | Likely Cause | Action | |-------|-------------------|-------------|--------| | Normal (bell curve) | Symmetric, single peak centered | Process is stable and predictable | Monitor. This is the target state. | | Shifted normal | Bell curve off-center toward one limit | Process mean has drifted | Re-center: adjust calibration, tool offset, or recipe | | Skewed right | Tail extends toward higher values | Physical constraint on the low end, or log-normal process | Check if the spec is appropriate. Some measurements (like response time) are naturally skewed. | | Skewed left | Tail extends toward lower values | Physical constraint on the high end | Same as above, opposite direction | | Bimodal (two peaks) | Two humps instead of one | Two populations mixed | Separate data by station, operator, shift, or component lot. Each population may be fine on its own. | | Truncated | Distribution cut off sharply at one end | Parts are being screened, sorted, or the measurement saturates | Check if screening is intentional. If it is, standard Cpk formulas may overestimate capability. | | Flat (uniform) | No clear peak, roughly even across range | Process is not controlled | Major investigation needed. Something is varying widely without control. | | Comb (alternating high/low bins) | Jagged pattern | Measurement resolution too coarse, or rounding | Increase instrument resolution or check data rounding | ## The Normal Curve Overlay TofuPilot overlays a normal distribution curve (Gaussian) on the histogram. This overlay uses the calculated mean and standard deviation from your data. When the histogram bars closely follow the curve, your data is approximately normal and standard capability formulas (Cp, Cpk) apply directly. When the bars diverge from the curve (heavy tails, skew, multiple peaks), the standard formulas may not accurately represent your process. The histogram makes this visible at a glance. ## Specification Limits on the Histogram TofuPilot draws USL and LSL as vertical lines on the histogram when limits are defined. This shows you: - How much of the distribution falls within spec - Whether the distribution is centered between the limits - How much margin exists on each side If the distribution tails extend past a spec line, some units are failing on that side. The visual makes it obvious which side needs attention. ## Mean and Sigma Lines The histogram also shows the process mean (center line) and sigma bands (±1σ, ±2σ, ±3σ). In a normal distribution: | Range | Contains | |-------|----------| | ±1σ | 68.3% of data | | ±2σ | 95.4% of data | | ±3σ | 99.7% of data | If the 3σ lines are inside the spec limits, the process is capable (Cp > 1.0). If they extend past the spec limits, the process variation exceeds the spec window. ## Feeding Histogram Data to TofuPilot Histograms require numeric measurements with enough data points to form a meaningful distribution. Define measurements with limits in your test code. ```python filename="histogram_test.py" import openhtf as htf from openhtf.util import units from tofupilot.openhtf import TofuPilot @htf.measures( htf.Measurement("output_voltage") .in_range(minimum=3.25, maximum=3.35) .with_units(units.VOLT), htf.Measurement("clock_frequency") .in_range(minimum=7.99, maximum=8.01) .with_units(units.HERTZ), ) def measure_board(test): test.measurements.output_voltage = 3.301 test.measurements.clock_frequency = 8.002 def main(): test = htf.Test(measure_board) with TofuPilot(test): test.execute(test_start=lambda: "UNIT-0001") if __name__ == "__main__": main() ``` Open the Process Control page, select a numeric measurement, and the histogram appears alongside the control chart. The normal curve, spec limits, and mean line render automatically. Right-click on a histogram bin to select data points in that value range or filter the view. ## Using Histograms for Process Improvement Histograms are diagnostic tools. When you spot a non-normal shape, investigate: **Bimodal?** Split the data. Filter by station, operator, or batch in TofuPilot's sidebar. Each subset may form its own normal distribution, revealing the source of the split. **Skewed?** Check whether the measurement is inherently bounded (you can't have negative leakage current) or whether a process constraint is causing the asymmetry. **Shifted?** Compare to the target. If the peak is close to one spec limit, re-center the process before it starts producing failures. **Too wide?** The distribution fills the spec window. Reduce variation (better fixtures, tighter component specs, more stable environment) or widen limits if the product design allows it. ### Migrate OpenHTF Scripts to TofuPilot CLI URL: https://www.tofupilot.com/guides/migrate-openhtf-scripts-to-tofupilot-cli Move OpenHTF scripts from the legacy with TofuPilot wrapper to the TofuPilot CLI runner with deterministic deploys, station auth, and live UI. The TofuPilot CLI is the new way to run OpenHTF scripts on a station. It is the runtime behind Stations v2.0, and it replaces the `with TofuPilot(test):` wrapper you used before. This guide shows what changes in your script and how to migrate. > **Starting fresh? Try a clean deploy first.** Before touching your existing script, create a new procedure in the dashboard and clone the [tofupilot/template-openhtf-starter](https://github.com/tofupilot/template-openhtf-starter) template from the new procedure flow. You will go from zero to a working live run in a few minutes, validate the new station setup end to end, and only then migrate your real script. ## Deprecation timeline The legacy operator UI is being deprecated. Target date is **July 1st, 2026**. Upcoming Python client releases will drop **real-time streaming** from the `with TofuPilot(test):` wrapper: the wrapper itself will keep working for post-run upload, but live phase, measurement, log, and attachment streams to the dashboard will only run through the new CLI. Our documentation is being updated to remove every reference to the legacy real-time path. We are reaching out directly to every team still using it to provide guidance and hands-on migration support. Once every active team has migrated, the legacy real-time server will be shut down. After that point only the new CLI-based stations will stream runs live. ## Why move to the CLI Stations v2.0 is powered by the TofuPilot CLI and brings a few things the old wrapper could not: - **One-command install on the station.** No more API key juggling, no manual Python setup. The CLI installs in one shell command from the station setup page and logs in once. - **Push from your repo, deploy automatically.** Connect the procedure to its GitHub repo and pick a branch (typically `main`) for auto-deploy. Pushes to that branch deploy to the right stations on their own. Other branches stay manual: deploy them from the dashboard when you are ready. - **Continuous run execution.** Stations loop runs back-to-back on their own. Operators scan a serial number, the run finishes, the next one is queued. No `while True:` loop in your script, no manual relaunch between units. - **Three UIs out of the box.** A local **kiosk UI** runs on the station for shop-floor operators and works fully offline. A **terminal UI** runs in your shell when an engineer drives a benchtop test. The **web operator UI** in the dashboard streams the same run live whenever the station is online, just like before. Customer feedback on the new operator UI has been the strongest of any release this year. - **Offline-friendly upload queue.** Runs are queued locally on the station and uploaded as soon as the network is back, so a flaky link never costs you data. - **Operator role.** A new role for shop-floor users that only sees the stations on its team, lands on a dedicated `/operator` page, and gets every run auto-attributed to the right person. No shared API key. - **Self-hosted ready.** The new CLI works against self-hosted TofuPilot deployments out of the box. The legacy operator UI never supported self-hosted, so this is the first time on-prem teams get the live operator experience. The CLI keeps every feature the previous wrapper shipped: identify-unit, validators, attachments, prompts, logs, post-run upload. ## Migration steps ### 1. Update your script (2 minutes) Drop the `tofupilot` import and the `with` block. Drop the procedure ID, batch, and any custom serial-number prompt too: the CLI handles unit identification (serial number, part number, revision, batch) natively through the operator UI before each run. ```python filename="main.py" from tofupilot.openhtf import TofuPilot # [!code --] import openhtf as htf def get_serial(): # [!code --] return input("Scan serial number: ") # [!code --] def measure_voltage(test): test.measurements.voltage = 3.3 test = htf.Test( # [!code --] measure_voltage, # [!code --] procedure_id="FVT1", # [!code --] test_description="Voltage check", # [!code --] ) # [!code --] test = htf.Test(measure_voltage) # [!code ++] with TofuPilot(test, batch_number="B-2024-09"): # [!code --] test.execute(get_serial) # [!code --] test.execute() # [!code ++] ``` Add a `pyproject.toml` next to it so the CLI can resolve dependencies: ```toml filename="pyproject.toml" [project] name = "fvt1" version = "0.1.0" dependencies = ["openhtf>=1.6"] ``` ### 2. Connect the procedure to your repository On the procedure page in the dashboard, link the GitHub repository that holds your script. As soon as it is linked, every push to the auto-deploy branch creates a deployment ready to roll out to your stations. Pushes to other branches stay manual and ship from the dashboard on demand. ### 3. Set up the station with the install command On the station detail page in the dashboard you will find the install command for Linux, Windows, and macOS, pre-filled with the right token. Run it on the station once. The CLI installs, registers itself with your organization, and the station shows up online in the dashboard. ### 4. Tune the station config From the station settings, pick the behaviors that match your floor: - **Launch on boot** so the station is ready as soon as it powers up. - **Kiosk UI** for full-screen operator mode, or **Terminal UI** for an engineer benchtop. - **Desktop shortcut** for one-click launch. - **Auto-update** to keep the CLI current between runs. You are done. The next operator-triggered run lands in the dashboard live, with attribution, validators, and attachments handled by the CLI. ### Why TofuPilot Needs Administration: write URL: https://www.tofupilot.com/guides/why-tofupilot-needs-administration-write Learn why the TofuPilot GitHub App requests Administration: write, what it enables, and how to scope it safely with selected repositories. # Why TofuPilot Needs Administration: write The TofuPilot GitHub App now requests the `Administration: write` repository permission. This guide explains why we ask for it ahead of an upcoming feature, what it enables, our security model, and the alternatives if you'd rather not grant it. ## Why the new permission The upcoming Stations v2.0 ships a new procedure-creation workflow that takes you from procedure to deployed-on-station in two steps. Since many users don't have a repository yet, or are new to TofuPilot Framework / OpenHTF, we'll provide ready-to-go templates that get cloned into your account before deploying. GitHub requires `Administration: write` for any third-party app to create a repository on a user's behalf. We have no other write operations beyond this one. The same permission is requested by leading developer platforms for the same reason — it's the only mechanism GitHub exposes for creating a repository from a template. ## Security model We take this permission seriously. Our protections include: - **No other write routes.** Our APIs expose no GitHub write operations beyond this single template-clone call, scoped to creating one new repository. - **Strict tenant isolation.** Connection records are isolated per organization at the database level — even an authenticated user from one organization cannot read another organization's data. - **Restricted to owners and admins.** Managing connections (create / update / delete) is restricted to organization owners and admins. - **Secrets kept out of the database.** The GitHub App private key required to mint installation tokens lives in our cloud provider's encrypted secret store, not in our application database. A database leak alone does not grant access to your GitHub account. ## If you'd rather not grant it You have three options: ### 1. Don't accept the permission The Clone Template feature won't work, but everything else (importing existing repos, auto-deploy, webhooks, operator UI) continues to work normally. You can create a repository manually on GitHub, then use the **Import** flow. ### 2. Scope to selected repositories When re-accepting the permissions, pick **Only select repositories** instead of **All repositories**. This limits the App's access to repositories you explicitly pick. After we clone a template, you may need to manually add the new repository to the App's allowed list before auto-deploy starts working on it. ### 3. Self-hosted or dedicated managed deployment Configure your own GitHub App with whatever permissions fit your security policy. TofuPilot adapts to the permissions available: features that need missing permissions are hidden, everything else continues to work normally. ## How to grant or scope the permission 1. In TofuPilot, open your organization's **New Procedure** page. If the permission is missing, an in-app banner will appear with a **Grant permission on GitHub** button once the feature ships. 2. Click the button. You're redirected to your GitHub installation's permissions page. 3. Choose **Only select repositories** if you want to scope the App tightly. Otherwise, **All repositories** is fine. 4. Click **Accept new permissions**. You're returned to TofuPilot, and the Clone Template form unlocks automatically. Approval takes about 30 seconds. ## What we use this permission for We use `Administration: write` only to call GitHub's "create a repository from a template" API during the Clone Template flow. Every API call is logged in your GitHub audit log so you can verify exactly what we do. We never delete, transfer, archive, or modify settings on existing repositories. ## Frequently asked questions ### Can you implement template clone without this permission? Not on GitHub. GitHub does not offer a narrower scope. Every platform that lets you clone a template into a new repository requests the same permission for the same purpose. ### What happens if I revoke the permission later? Revoke it whenever you want from your GitHub install settings. Clone Template stops working immediately; everything else keeps working normally. Repositories created before the revoke are unaffected. ### Does this give TofuPilot access to my private code? Only for repositories the App is installed on. With **Only select repositories**, you control exactly which ones. With **All repositories**, the App sees everything in the account it's installed on. Read access to repository contents is governed by a separate permission TofuPilot has had since the beginning — `Administration: write` is solely about creating new repositories. ### Operator Interface with OpenHTF URL: https://www.tofupilot.com/guides/operator-interface-with-openhtf Build the operator interface for an OpenHTF test on a TofuPilot station: text prompts, confirm buttons, text input, and live images. # Operator Interface with OpenHTF OpenHTF can pause a test to interact with the operator through its `user_input` plug. When the test runs on a TofuPilot station, these prompts render in the station kiosk: an optional image, your message, and either a confirm button or a text field. This guide covers that full surface with examples, and ends with how it looks on the station. ## What the Operator Sees A prompt appears on one screen with: - An optional image, above the message - The message text - A Continue button, or a text field, depending on the call The operator never opens a terminal or a separate viewer. ## Prerequisites - Python 3.10+ - OpenHTF installed (`pip install openhtf`) ## The UserInput Plug A single call drives the prompt. Attach the `UserInput` plug to a phase and call `prompt()`. ```python filename="signature.py" from openhtf.plugs import user_input prompts.prompt( message, # text shown to the operator text_input=False, # False = confirm button, True = text field timeout_s=None, # optional, raises PromptUnansweredError on timeout image_url=None, # optional image shown inline in the prompt ) ``` It returns the operator's text (an empty string when `text_input=False`). ## Message and Continue Button ```python filename="power_cycle.py" import openhtf as htf from openhtf.plugs import user_input @htf.plug(prompts=user_input.UserInput) def power_cycle(test, prompts): prompts.prompt("Power-cycle the unit, wait for the LED, then click Continue.") ``` ## Text Input ```python filename="scan_serial.py" @htf.plug(prompts=user_input.UserInput) def scan_serial(test, prompts): serial = prompts.prompt("Scan the serial number:", text_input=True) test.dut_id = serial ``` ### Yes / No Decisions OpenHTF has no two-button Yes/No widget. The pattern is always the same: ask for a typed answer, then branch on it, usually returning `PhaseResult.CONTINUE` to go on or `PhaseResult.STOP` to halt the test. The example below gates the test on an LED check. ```python filename="led_check.py" @htf.plug(prompts=user_input.UserInput) def led_check(test, prompts): answer = prompts.prompt("Does the LED blink green? Type y or n:", text_input=True) return htf.PhaseResult.CONTINUE if answer.strip().lower() == "y" \ else htf.PhaseResult.STOP ``` The same shape fits many operator decisions, for example: confirming a connector is fully seated before applying power, judging a visual pass or fail on a finish or a label, or deciding whether a unit goes to rework after an inspection. ## Show an Image in the Prompt Pass `image_url` and the image renders inline, above the message, live during the run. It works with a confirm button and with a text field. The URL is anything an HTML image tag accepts. ### Hosted image ```python filename="connector_check.py" @htf.plug(prompts=user_input.UserInput) def connector_check(test, prompts): prompts.prompt( "Connect the cable as shown, then click Continue.", image_url="http://localhost:8080/reference/connector.png", ) ``` ### Local image as a data URI (no server needed) On a station this is usually the simplest: read a local file and inline it as a base64 data URI, so there is nothing to host. ```python filename="visual_inspection.py" import base64 def data_uri(path, mime="image/png"): with open(path, "rb") as f: return f"data:{mime};base64," + base64.b64encode(f.read()).decode("ascii") @htf.plug(prompts=user_input.UserInput) def visual_inspection(test, prompts): reference = data_uri("/opt/station/reference/board_top.jpg", "image/jpeg") answer = prompts.prompt( "Does the board match the reference? Type y or n:", text_input=True, image_url=reference, ) return htf.PhaseResult.CONTINUE if answer.strip().lower() == "y" \ else htf.PhaseResult.STOP ``` ### Image captured during the test ```python filename="confirm_capture.py" @htf.plug(prompts=user_input.UserInput) def confirm_capture(test, prompts): path = "/tmp/capture.png" camera.grab_frame(path) # your capture code prompts.prompt( "Is the captured image in focus and centered?", text_input=True, image_url=data_uri(path, "image/png"), ) ``` Notes: use `image/png` or `image/jpeg`; the URL must load from the browser running the kiosk, so a data URI is the safest on a station. `image_url` shows the image live; to also keep it in the run record, attach it with `test.attach_from_file(path)`. ## Live Status Text OpenHTF has no self-updating console line for the operator. To surface progress during a phase, use `test.logger`: each call emits a log record live, streamed to whatever operator UI is showing the run (OpenHTF's own web GUI, or the TofuPilot kiosk) and kept in the test record. Use it for progress, then a short prompt once the step is done. ```python filename="discharge.py" @htf.plug(prompts=user_input.UserInput) def discharge(test, prompts): test.logger.info("Waiting for capacitor to discharge...") # live log entry wait_for_discharge() # your code prompts.prompt("Capacitor discharged. Click Continue.") ``` ## What the Operator Sees in TofuPilot These prompts render in the TofuPilot station kiosk, the operator UI the CLI serves locally and opens in a browser. Enable the kiosk on the station, or force it for a single run with `tofupilot run --kiosk`. When a phase calls `prompt()`, the kiosk shows the image, the message, and the Continue button or text field on one screen; the operator responds and the run continues. To watch from the dashboard, open the station's operator view at `//operator/`. If you need richer inputs than these OpenHTF prompts offer, such as dropdowns, checklists, sliders, switches, or a live progress bar, you can add them with a TofuPilot framework procedure that declares UI components. ### Migrate a Station from v1.0 to TofuPilot CLI URL: https://www.tofupilot.com/guides/migrate-a-station-from-v10-to-tofupilot-cli Move a test bench from the v1.0 desktop station app to the TofuPilot CLI: one-command install, automatic deploys, and a live operator UI. The TofuPilot CLI is the new way to run a test station. It is the runtime behind Stations v2.0, and it replaces the v1.0 station: the desktop app that ran on the bench, pulled deployments, and executed procedures locally. The CLI does the same job with one-command install, automatic deploys from Git, and the same live operator UI. This guide shows how to move a bench over. > On a self-hosted instance? Set up the deployer first so your instance can build station bundles, and enable the realtime server so runs stream live. Follow [Enable Realtime and Deploys on Self-Hosted](/guides/enable-realtime-and-deploys-on-self-hosted), then come back here. Cloud TofuPilot has both on already, so you can start right away. ## Why move to the CLI Station v1.0 was a desktop application installed on each bench. Stations v2.0 run on the TofuPilot CLI, which keeps everything the desktop app did and adds what it could not: - **One-command install.** No installer to download per machine. The CLI installs in one shell command from the station setup page and registers itself on first boot. - **Deploy from Git automatically.** Connect the procedure to its GitHub repo and pick a branch (typically `main`) for auto-deploy. Pushes to that branch roll out to the right stations on their own. Other branches stay manual: deploy them from the dashboard when you are ready. - **Remote and kiosk operator UI.** Drive a run from the dashboard over the web, or from a full-screen local kiosk UI on the bench. A terminal UI is there too when an engineer drives a benchtop test. - **No Git provider login on the bench.** The desktop app synced procedure code straight from GitHub or GitLab, so each bench needed Git provider access. Linking the repo now happens once in the dashboard, and the bench pulls a pre-built artifact instead, so it never touches your repository and you control deploys entirely from the web. - **Offline-friendly upload queue.** Runs queue locally on the bench and upload as soon as the network is back, so a flaky link never costs you data. - **Runs headless anywhere.** The desktop app needed a desktop environment. The CLI runs headless on a server, a rack box, or a small single-board computer, and works against self-hosted deployments out of the box. ## Migration steps ### 1. Make sure the procedure has a deployment The CLI runs from a built artifact, so the procedure needs a fresh deployment for the new station to pull. On the procedure page in the dashboard: - If the procedure is already linked to its Git repository (most v1.0 procedures are), trigger a new deployment: push to the auto-deploy branch, or deploy manually from the dashboard. - If it is not linked yet, link the GitHub or GitLab repository first, then deploy. That build is what the new station pulls and runs. ### 2. Install the CLI on the bench Open the station's detail page in the dashboard and copy its install command. It is pre-filled with the station's setup token, and on self-hosted it also carries the `--url` flag pointing at your instance, so copy it from the dashboard rather than typing it by hand. It looks like this: ```bash title="install" curl -fsSL tofupilot.sh/install | sh -s -- --token ``` Run it on the bench once. The CLI installs, registers itself with your organization, and the station shows up online in the dashboard. The setup token expires one hour after issue, so if it lapses, generate a new one from the station's row. ### 3. Tune the station config From the station settings, pick the behaviors that match your floor: - **Launch on boot** so the station is ready as soon as it powers up. - **Kiosk UI** for full-screen operator mode, or **Terminal UI** for an engineer benchtop. - **Desktop shortcut** for one-click launch. - **Auto-update** to keep the CLI current between runs. ### 4. Retire the desktop app Once the bench is running runs through the CLI and they land in the dashboard live, uninstall the v1.0 desktop app from the machine. Station v1.0 is being phased out, and we are reaching out to every team still on it with hands-on migration support. You are done. The next operator-triggered run lands in the dashboard live, with validators and attachments handled by the CLI. ### Why & How to Run Test Stations Without Root URL: https://www.tofupilot.com/guides/why-how-to-run-test-stations-without-root Running a test station as root is a costly shortcut. Learn least-privilege setup on Linux and Windows: low-privilege account, scoped hardware access, a service. # Why and How to Run Test Stations Without Root Access Running a production test station as `root` (Linux) or Administrator (Windows) is the fastest way to get hardware access working, and a shortcut you pay for later in security exposure and operational fragility. The highest-value habit is simple: **don't log the station in as an admin.** This guide covers that cheap win first, then how far you can take least privilege on Linux and Windows without breaking hardware access or boot-time startup. ## The Shortcut and What It Costs When the priority is to deploy fast, running the whole station as root is often the shortest path: a device throws a `Permission denied`, and elevating unblocks it right away. Over time it hardens into one of these: | Shortcut you see on the floor | What it saves | What it quietly costs | |---|---|---| | Auto-login as root, no password | Operator opens the session instantly | Anyone with physical access owns the machine | | Root password on a sticky note | No one waits to unlock | Same as no password, plus a paper trail to your secret | | `sudo` calls inside the runtime code | Script "just works" | Every run holds full-system power it never needs | | `NOPASSWD` in sudoers for the runner | No prompts during a shift | Any bug or misuse escalates silently | Sometimes it is a shortcut under deadline; sometimes a vendor tool genuinely demands root and there's no clean alternative that day. Either way, the cost is the same and it compounds across every station, every audit, and every machine that leaves the building for repair. The fix is not "never elevate." It is: **elevate to install, never to run.** Root is fine for setup (installing software, drivers, device rules), not for the identity that executes tests shift after shift. ## The Cheap Win: Don't Log In as Admin Before any of the advanced setup below, one change removes most of the risk for almost no effort: **the operator logs in as a standard, non-admin user, and the station runs in kiosk mode.** This alone is the 80/20. | Risk | A standard account + kiosk removes it? | |---|---| | Operator "borrows" the station to browse, copy files, install a tool | Yes, no admin rights, and the kiosk hides the desktop | | Operator changes a system setting they shouldn't | Yes, they can't, by design | | A bug in a parsed file corrupts the whole OS | Mostly, the process can't write outside the user's reach | | Stolen/repaired machine leaks every stored secret | Reduced, the account's reach bounds what's exposed | A standard account protects the operator from the machine as much as the machine from the operator: they should physically not be able to do what they aren't meant to. Kiosk mode is covered separately; the point here is that the account it runs under should not be an admin. Everything below is how to keep it that way while still reaching hardware and starting on boot. ## The Security Cost (Why This Matters) Least privilege is error containment: the software equivalent of fusing a circuit instead of trusting nothing will ever short. A test runner parses files it did not write (firmware images, calibration data, serial responses, config from a share), so any bug runs with the identity of the process. | Risk | As a low-privilege account | As root / admin | |---|---|---| | **Blast radius** | A bad parse corrupts one working directory | The same bug can overwrite the OS or another line's data | | **Credential exposure** | Process reads only its own API key | Process can read every secret on the machine | | **Supply chain / privilege escalation** | A compromised dependency is contained | A single bad package owns the box, no escalation needed | | **Lost or repaired machine** | Data at rest is bounded by the account's reach | Full disk and every stored credential are exposed | | **Audit** | Logs attribute actions to a station identity | Logs say `root did X` for everything, unattributable | The escalation risk is concrete enough that good tooling refuses to enable it: the TofuPilot CLI will not install a root boot-service whose binary lives in a user-writable directory, because a non-root user who could overwrite that binary would gain root at the next boot. The audit line is now a business consequence too. Standing local admin on floor workstations is increasingly a **CMMC / NIST 800-171 audit failure**, a real barrier for anyone in aerospace, defense, or high-end EMS supply chains. And the blast-radius line is what security teams call **lateral movement**: an attacker who lands on one over-privileged station moves across the whole plant before anyone notices. ## "But the Station Is Offline" Root-on-benches is often defended with "the station has no internet, so who cares." That misreads what offline means on a factory floor. The station is air-gapped from the *internet*, but it usually sits on a **secured local network**, reachable from other machines and often able to reach the MES and the production server where centralized production data lives. An IT team only accepts the painful drawback of an offline station when the machines and data on that local network are sensitive enough to justify it. In other words, "offline" is a signal that security matters *more* here, not less. A station running as root on that network is a soft entry point to everything else on it: exactly the lateral movement the isolation was meant to prevent. Removable media compounds it: an offline station still takes firmware images on USB sticks and still leaves the building for repair (the Stuxnet worm reached isolated industrial systems precisely by USB). ## The Principle: Elevate to Install, Never to Run | Task | Needs elevation? | Who runs it | |---|---|---| | Install software, drivers, device rules | Yes, once | Admin, at setup | | Read/write a serial or USB device at runtime | No | Low-privilege account via group / rule | | Execute tests, upload results | No | The station's non-admin account | | Flash firmware via raw I/O | Sometimes | Grant the one capability, not full root | The rest of this guide is how to build that on Linux and Windows. ## Linux: Least-Privilege Setup ### Step 1: Run Under a Low-Privilege Account The account the station runs under should be a normal user with no admin rights: not `root`, not a member of `sudo`/`wheel`. On most benches this is the `operator` account the kiosk already uses. ```bash filename="01-create-operator-user.sh" # A normal low-privilege account for the station. NOT in sudo/wheel/adm. # The operator can log in and run the kiosk UI, but can't touch the system. sudo useradd --create-home operator id operator # confirm: not a member of sudo, wheel, or adm ``` ### Step 2: Grant Hardware Access With Groups, Not Root The reason people reach for root is device permissions. The correct fix is group membership: add the operator account to the group that already owns the device. ```bash filename="02-grant-hardware-access.sh" # Serial ports (/dev/ttyUSB*, /dev/ttyACM*) are group-owned by 'dialout' sudo usermod -aG dialout operator # Hot-plugged USB instruments are usually group-owned by 'plugdev' sudo usermod -aG plugdev operator # Confirm which group owns your device (4th column) ls -l /dev/ttyUSB0 # crw-rw---- 1 root dialout 188, 0 ... /dev/ttyUSB0 # ^^^^^^^ group that grants access ``` For a device not owned by a standard group, write a **udev rule** (udev is the Linux subsystem that assigns ownership and permissions to devices as they appear): ```text filename="/etc/udev/rules.d/99-station-dut.rules" # Match a specific instrument by USB vendor/product ID and hand it to # the 'dialout' group (which the operator is in). No root at runtime. SUBSYSTEM=="tty", ATTRS{idVendor}=="0403", ATTRS{idProduct}=="6001", GROUP="dialout", MODE="0660" ``` ```bash filename="03-reload-udev.sh" sudo udevadm control --reload-rules && sudo udevadm trigger ``` For a binary that needs one specific kernel privilege (e.g. raw I/O to flash firmware), grant that one **capability** instead of root. Capabilities split root's powers into independent units: ```bash filename="04-grant-capability.sh" # Give only raw-I/O to this one binary, nothing else root can do sudo setcap cap_sys_rawio+ep /usr/local/bin/flash-tool getcap /usr/local/bin/flash-tool ``` ### Step 3: Launch on Boot Without Root This is where root usually sneaks back in. When you enable **Launch on boot**, the TofuPilot CLI installs the right startup service automatically, so you don't write a unit file by hand: | CLI runs as | Service it installs | Kiosk | |---|---|---| | `operator` (non-admin) | systemd **user** service (`~/.config/systemd/user/`) | Yes, on the local display | | `root` | systemd **system** service (`/etc/systemd/system/`) | No, headless, driven from the dashboard | The trap that pushes people to root: if you enable launch-on-boot over SSH as the operator (a shell with no login session), `systemctl --user` has no session bus and fails: ```text filename="journalctl-error.txt" Failed to connect to bus: No medium found systemctl --user: Failed to connect to user scope bus ``` The fix is **not** to switch to root. It is to let the operator's user services run without an interactive login. That capability is called **lingering** in systemd (like a machine that keeps its background routine running even when no one is at the HMI): ```bash filename="05-enable-linger.sh" # Operator's systemd services now start at boot with nobody logged in sudo loginctl enable-linger operator loginctl show-user operator --property=Linger # -> Linger=yes ``` With lingering on, the non-root station starts at every boot, keeps its kiosk display, and never needs root for startup. Credentials follow the same account: run `tofupilot login --token ` as the operator, not `sudo`. ### Step 4: Verify Least Privilege ```bash filename="06-verify.sh" # The station process should show 'operator', not root ps -o user,cmd -C tofupilot # Run the check AS operator (sudo -u switches identity, it does not elevate) sudo -u operator cat /dev/ttyUSB0 # opens via the dialout group, not "Permission denied" ``` ## Windows: Least-Privilege Setup Same principle: the operator account is a **standard user**, not Administrator. Windows makes most of this easy because standard users already have the access a test runner needs. | Need | Least-privilege grant | Avoid | |---|---|---| | Read/write a COM port | Standard users can open COM ports by default; no admin needed | Running as admin "to be safe" | | USB instrument via vendor driver | Install the driver once as admin; runtime access is per-device | Admin at runtime | | Write logs / results | Grant the account write access to one folder (e.g. `C:\tofupilot`) | Writing under `C:\Program Files` | | Launch on boot | TofuPilot registers a per-user `Run` entry, fires at the operator's logon, no admin | A machine-wide service running as SYSTEM/admin | ```powershell filename="01-create-operator-user.ps1" # Local standard user (NOT added to Administrators) $pw = Read-Host -AsSecureString "Password for the operator account" New-LocalUser -Name "operator" -Password $pw -PasswordNeverExpires ` -Description "Test station operator" # Deliberately NOT: Add-LocalGroupMember -Group "Administrators" ``` ```powershell filename="02-grant-folder-access.ps1" # Give the operator account its own writable working directory New-Item -ItemType Directory -Path "C:\tofupilot" -Force icacls "C:\tofupilot" /grant "operator:(OI)(CI)M" ``` On Windows the station runs as a per-user logon process under the standard operator account, so it starts when the operator logs in, with no admin service and no SYSTEM privileges. Store the API key in the operator's user-scoped environment or a file under `C:\tofupilot` with permissions limited to that account, never machine-wide. ## When Root Is the Pragmatic Choice Best practice sometimes conflicts with productivity, and the person who knows the constraints, the developer at the bench during a production rush, is the one who should decide. At TofuPilot we don't block that decision: the CLI runs under root or admin at runtime if you choose to, because removing deployment friction during a rush matters more than enforcing rigidity from the outside. Running the CLI as root even installs the correct system service for you. If you do run as root, do it deliberately and shrink the risk: | If you run as root | Reduce the risk by | |---|---| | A vendor tool or raw I/O genuinely needs it | Scope it: `setcap` on that one binary instead of root everywhere | | Hardware access is blocking a deadline | Ship the root shortcut, then schedule the udev/group fix as follow-up | | Headless bench, launch on boot | Keep the binary in `/usr/local/bin` owned by root (the CLI refuses a user-writable one) | | Physical machine can be stolen or sent for repair | Encrypt the disk; keep the API key scoped and revocable | | Operators share the station | Pair root runtime with kiosk lockdown so they can't wander off the test UI | The point is not purity. It is knowing what the shortcut costs so the choice is informed, and moving to least privilege once the fire is out. ## Troubleshooting | Issue | Cause | Fix | |---|---|---| | `systemctl --user: Failed to connect to user scope bus` | Enabling launch-on-boot from a shell with no login session (SSH) | `sudo loginctl enable-linger operator`, don't switch to root | | Station doesn't start at boot, no error | User service installed but lingering is off | Enable lingering (above); confirm with `loginctl show-user operator` | | `Permission denied` on `/dev/ttyUSB0` after dropping root | Operator not in the device's group | `usermod -aG dialout operator`, then log out/in or reboot | | Device works for root but not the operator | udev rule not applied, or wrong vendor/product ID | Re-check IDs with `lsusb`, reload with `udevadm control --reload-rules && udevadm trigger` | | CLI refuses to install a root system service | Binary sits in a user-writable path (a root-at-boot escalation vector) | Move it to `/usr/local/bin` owned by root, then re-run | | Windows runner can't write results after dropping admin | Writing to a protected path | Point output at the operator's own folder (`C:\tofupilot`) | ### Realtime and Deploys on Self-Hosted TofuPilot URL: https://www.tofupilot.com/guides/enable-realtime-and-deploys-on-self-hosted Add the Centrifugo realtime server and the Docker-in-Docker deployer to a self-hosted TofuPilot instance for live status and Git-push deployments. Self-hosted TofuPilot ships two services that bring it to cloud parity: a **realtime server** and a **deployer**. Realtime flips stations online the moment they connect and streams runs live; the deployer builds your procedure bundle from Git and streams the build log while it runs. Both come down with the deploy script. This guide shows what they need and how to confirm they are working. This is for **existing self-hosted instances** adding these services. Both landed in **2.33**, so if you are upgrading from an earlier version they are new to you: re-running the deploy script pulls them in, but the realtime server also needs a DNS record that older instances never had (Step 1). Cloud TofuPilot (`tofupilot.app`) has both on already, and fresh self-hosted instances now get them by default from the [Configure](/docs/self-hosting/configure) and [Deploy](/docs/self-hosting/deploy) reference docs. For the general upgrade flow, see [Upgrade](/docs/self-hosting/upgrade). Once realtime and the deployer are running, you can move your benches off the v1.0 desktop app with [Migrate a Station from v1.0 to TofuPilot CLI](/guides/migrate-a-station-from-v10-to-tofupilot-cli). ## What you get - **Realtime.** Live station status, telemetry, and run streaming to the operator UI and dashboard, plus streaming build logs. Without it the app still works, but live views fall back to manual refresh. - **Deployer.** The Git-push deployment pipeline: link a procedure to a repo, push, and the bundle builds and rolls out to your stations. Without it the rest of the stack works, but the build pipeline is unavailable, so Stations v2.0 cannot get their bundles. ## Step 1: Add the realtime subdomain A self-hosted instance needs three subdomains. The third carries the realtime traffic. Create a DNS **A record** pointing at your server: ```text title="DNS" realtime.tofupilot.yourcompany.com A ``` In `.env`, the realtime host defaults to `realtime.`, so if your app runs at `tofupilot.yourcompany.com` you can leave it unset. To use a different host, set `CENTRIFUGO_DOMAIN_NAME`. The deploy script generates the realtime and deployer secrets on first run and reuses them after, so you do not manage them by hand. ## Step 2: Run Docker as root (for the deployer) The deployer builds in a sandbox that needs a privileged container, which only standard (root) Docker allows. Under rootless Docker the deployer stays down and the build pipeline is unavailable, while everything else, including realtime, works normally. If you need the build pipeline, install Docker the standard way, not rootless. ## Step 3: Deploy Re-run the deploy script. The license key is already in `.env`, so you do not pass it again: ```bash title="deploy" curl -fsSL https://tofupilot.sh/deploy | bash ``` The script brings up the realtime and deployer containers alongside the rest of the stack and requests a TLS certificate for the realtime subdomain automatically. Confirm everything is running: ```bash title="check containers" docker compose ps ``` ## Step 4: Verify **Realtime:** open the dashboard and watch a station's status, or trigger a run and watch phases stream. Status should flip and runs should update with no page refresh. **Deployer:** link a procedure to a Git repo, push to the auto-deploy branch, and open the deployment. The build log should scroll live as the bundle builds, then the artifact rolls out to your stations. If realtime stays disconnected, the cause is almost always the subdomain: confirm the `realtime.` A record resolves to your server, its TLS certificate issued, and port 443 is open for it just like the app. ### TofuPilot Framework vs OpenHTF URL: https://www.tofupilot.com/guides/tofupilot-framework-vs-openhtf How the TofuPilot Framework and OpenHTF differ, and how each one plugs into the TofuPilot dashboard and stations for live test data. OpenHTF and the TofuPilot Framework are the two open-source Python frameworks purpose-built for hardware manufacturing test. OpenHTF was built by Google in 2016 and is the established, widely deployed option. The TofuPilot Framework is newer, built on a Rust execution engine for parallel execution. Both model a test as ordered phases with structured measurements, both connect instruments through reusable plugs, and both run natively on the TofuPilot CLI and stream to the same dashboard. This guide compares them side by side with real code so you can pick the right one. ## Framework Overview | | TofuPilot Framework | OpenHTF | |---|---|---| | **Language** | Python (Rust engine) | Python | | **License** | MIT | Apache 2.0 | | **Maintainer** | TofuPilot | Google | | **First release** | 2025 | 2016 | | **Authoring model** | Declarative YAML procedure plus Python phases and plugs | Code-first Python with decorators | | **Execution** | Parallel phases and slots | Sequential phases | | **Community** | Newer, smaller | Established, larger install base | ## Feature Comparison | Feature | TofuPilot Framework | OpenHTF | |---|---|---| | **Phases** | Yes | Yes | | **Numeric, string, boolean measurements** | Yes | Yes | | **Logs and attachments** | Yes | Yes | | **Unit identification by serial number** | Yes | Yes | | **Sub-units (child components)** | Yes | Yes | | **Plugs (shared instrument connections)** | Yes | Yes | | **Multi-dimensional measurements (waveforms, sweeps)** | Yes | Yes | | **Aggregations (mean, min, max, std over arrays)** | Yes | No | | **Per-axis validators (validate curve shape)** | Yes | No | | **Operator prompts** | Yes | Yes | | **Custom forms and display components** | Yes | No | | **Retry, timeout, setup/teardown** | Yes | Yes | | **Conditional execution** | Yes | Yes | | **Parallel phases** | Yes | No | | **Multi-slot execution (multiple boards on one fixture)** | Yes | No | On the shared data model the two are close to parity. The gaps fall in three areas: advanced measurement analysis, the operator UI, and execution. ## The Same Test in Both Frameworks A functional test that verifies a board's 3.3V rail and firmware version, written in each framework. ### TofuPilot Framework Version The procedure YAML declares the sequence and limits. The phases and plug are plain Python that the procedure references by name. ```yaml filename="procedure.yaml" name: Functional Test version: 1.0.0 unit: serial_number: default_value: "PCBA000001" part_number: default_value: "PCBA-100" plugs: - name: dut python: plugs.dut:Dut main: - name: Measure Voltage python: phases.measure_voltage measurements: - name: rail_3v3 unit: V validators: - operator: ">=" expected_value: 3.2 - operator: "<=" expected_value: 3.4 - name: Check Firmware python: phases.check_firmware measurements: - name: firmware_version validators: - operator: "==" expected_value: "2.1.0" ``` ```python filename="phases/measure_voltage.py" # Parameters are injected by name: measurements, plus the dut plug. def measure_voltage(measurements, dut): measurements.rail_3v3 = dut.read_voltage() ``` ```python filename="phases/check_firmware.py" def check_firmware(measurements, dut): measurements.firmware_version = dut.query_firmware() ``` ```python filename="plugs/dut.py" class Dut: """Manage the connection to the unit under test.""" def read_voltage(self) -> float: return 3.31 # Replace with an instrument read def query_firmware(self) -> str: return "2.1.0" # Replace with a DUT query ``` Run it from the procedure directory with `tofupilot run`, then deploy to a station with a Git push. ### OpenHTF Version OpenHTF attaches measurements and plugs to phase functions with decorators, then runs them in sequence. ```python filename="openhtf_test.py" import openhtf as htf from openhtf.plugs import BasePlug from openhtf.util import units class DutPlug(BasePlug): """Manage the connection to the unit under test.""" def read_voltage(self) -> float: return 3.31 # Replace with an instrument read def query_firmware(self) -> str: return "2.1.0" # Replace with a DUT query @htf.measures( htf.Measurement("rail_3v3").in_range(3.2, 3.4).with_units(units.VOLT) ) @htf.plug(dut=DutPlug) def measure_voltage(test, dut): test.measurements.rail_3v3 = dut.read_voltage() @htf.measures(htf.Measurement("firmware_version").equals("2.1.0")) @htf.plug(dut=DutPlug) def check_firmware(test, dut): test.measurements.firmware_version = dut.query_firmware() def main(): test = htf.Test(measure_voltage, check_firmware, part_number="PCBA-100") test.execute(test_start=lambda: input("Scan serial number: ")) if __name__ == "__main__": main() ``` The TofuPilot CLI runs both frameworks natively and streams phases, measurements, logs, attachments, and the operator UI to the dashboard, with no extra code on either side. ## Key Differences in the Code | Aspect | TofuPilot Framework | OpenHTF | |---|---|---| | **Sequence definition** | Declared in `procedure.yaml`, separate from logic | Function order in `htf.Test(...)`, in code | | **Limits** | `validators` in YAML, stored as data | `.in_range(3.2, 3.4)` chained in code | | **Plug injection** | By parameter name (`dut`) | By `@htf.plug(dut=...)` decorator | | **Measurement access** | `measurements.rail_3v3 = ...` | `test.measurements.rail_3v3 = ...` | | **Plug class** | Plain Python class | Subclass of `BasePlug` | ## Execution and Parallelism This is the clearest architectural difference. OpenHTF runs phases sequentially against one unit under test. To test multiple boards or run independent steps concurrently, you orchestrate that yourself outside the framework. The TofuPilot Framework runs on a Rust engine that executes independent phases in parallel and serializes only where you declare a `depends_on`. It also supports multi-slot execution, testing several boards in parallel on one fixture, which directly reduces cycle time on high-volume lines. Your phase and plug code stays plain Python; the engine handles the scheduling. If your bottleneck is throughput on a fixture that holds multiple DUTs, this is the deciding factor. If you test one unit at a time, it matters less. ## Measurements and Operator UI Both frameworks capture typed measurements with limits, units, and validators, and both support multi-dimensional measurements for waveforms and sweeps. The TofuPilot Framework adds two things OpenHTF does not: aggregations (mean, min, max, std computed over an array) and per-axis validators that check the shape of a curve rather than only its endpoints. For RF sweeps, thermal profiles, or any test where the waveform itself is the spec, that removes glue code. Both ship a built-in operator UI. OpenHTF provides web prompts for operator input such as scanning a serial number. The TofuPilot Framework adds declarative form and display components (text, number, switch, radio, select, multiselect, image choice, sliders, progress) that you define in the procedure without writing any frontend. ## When to Use the TofuPilot Framework The TofuPilot Framework is the better choice when: - **Your line is throughput-bound.** Parallel phases and multi-slot execution test several boards at once on one fixture and cut cycle time. OpenHTF executes sequentially. - **You validate waveforms or sweeps.** Built-in aggregations and per-axis validators check curve shape, not only endpoints, with no glue code. - **Operators need rich guided workflows.** Declarative forms and display components render on the station with no frontend work. - **You want the test plan separate from the logic.** The YAML procedure is readable and diffable independently of the Python phases. - **You are setting up new stations.** Deploy-on-push, real-time streaming, and offline queueing are built into the CLI. ## When to Use OpenHTF OpenHTF is the better choice when: - **You already run it.** Existing OpenHTF suites run natively on TofuPilot with no rewrite. The migration cost is not worth it. - **Maturity and community matter most.** Years of production use at scale and a larger body of public examples and patterns. - **You want a single pure-Python paradigm.** Everything is in Python, with no separate YAML layer to learn. - **Apache 2.0 fits your legal review.** Its explicit patent grant can matter in some procurement contexts. ## Using Both Together You do not have to choose the framework to choose the platform. A common path is to keep an existing OpenHTF suite running as is, and reach for the TofuPilot Framework on new stations that need parallel slots or richer measurements. Both stream phases, measurements, logs, attachments, and the operator UI to the same dashboard, and both are open-source, so neither choice locks the test code itself. ## Decision Matrix | Question | Better fit | |---|---| | Testing multiple boards on one fixture, throughput-bound? | TofuPilot Framework | | Validating waveform or sweep shape, not just endpoints? | TofuPilot Framework | | Need rich operator forms on the station? | TofuPilot Framework | | Already running OpenHTF suites in production? | OpenHTF | | Want a single pure-Python paradigm, no YAML? | OpenHTF | | Maximum maturity and community matter most? | OpenHTF | Both run on the TofuPilot CLI and upload to the same dashboard, so you can start with either and mix them per station. ### Track Retest Counts to Catch Rogue Units URL: https://www.tofupilot.com/guides/track-retest-counts-to-catch-rogue-units Learn how to spot rogue units early by tracking retest counts per serial number, with self-calibrating alerts on abnormal shop visits in TofuPilot. Some units never stay fixed. They come off the aircraft, test fine or get a minor repair, go back into service, and return again a few weeks later. The industry calls them rogue units, and they are expensive out of proportion to their numbers: each one cycles through removal, shipping, induction, a full bench test, and recertification on every loop. This guide shows how to catch them early using nothing but the retest count you already generate. ## What Makes a Unit Rogue Airlines and suppliers define rogues contractually. A common definition: the same serial number removed three or more times for similar discrepancies, or four no-fault-found shop visits within twelve months. Airbus documents a comparable policy for its suppliers, and its maintenance briefing notes describe a fuel quantity indicator that returned to the shop fifteen times before finally causing a low-fuel event in service. The failure behind most rogues is intermittent: a cracked solder joint, a chafed wire, connector fretting. It appears under vibration and thermal cycling on the aircraft and disappears on a warm, still bench. A conventional functional test scans one circuit at a time and can miss a fault lasting microseconds, so the unit passes, ships, and fails again. You cannot fix an intermittent you have not localized. But you can stop losing track of the units that carry one, and that is a data problem, not a hardware problem. ## Identify Every Unit by Serial Number Rogue tracking collapses if runs are not tied to unit identity. In a TofuPilot procedure, the unit block makes serial and part number a first-class part of every run: ```yaml filename="procedure.yaml" name: LRU Bench Acceptance Test version: 1.0.0 unit: serial_number: default_value: "LRU000042" part_number: default_value: "622-1234-001" ``` On a station, the operator scans or enters the serial number at the start of each run. Every test result, pass or fail, is then attached to that serial forever. ## Read the Unit History With identity in place, TofuPilot builds the per-unit history automatically. The unit page shows every run for a serial number: date, procedure, result, and every measured value. A rogue candidate reads like a story: - Visit 1: removed for reported fault, tests NFF, returned to stock. - Visit 2, five weeks later: same reported fault, tests NFF again. - Visit 3: fails marginally on one measurement, minor repair, released. - Visit 4: back again. On paper records spread over months, this pattern is invisible unless someone remembers the serial number. In a per-unit history it is one screen. The measurements add what the visit count alone cannot: a unit that tests NFF but whose signal margin shrinks on every visit is not fine, it is degrading. That trajectory is the difference between "no fault found" and "fault not yet caught". ## Alert on Abnormal Retest Rates Reviewing histories by hand only works for units someone already suspects. TofuPilot's retest threshold alert watches all of them: it fires when a unit has been tested more often than is normal for that procedure. The threshold self-calibrates, learning each procedure's ordinary retest rate, so a part that routinely needs two runs per visit does not generate noise while a genuine repeat offender does. The alert resolves manually once the unit's history has been reviewed, which fits how shops actually work: the point is to force a human look at the right serial numbers, early. ## From Detection to Action Once a serial number is flagged, the shop has options that all work better early: - **Deep test instead of another standard cycle.** Route the unit to environmental screening (temperature and vibration during test) or intermittence-focused equipment rather than repeating the same bench test that already passed it. - **Quarantine.** Hold the unit and monitor the aircraft for a few flights before deciding; if the fault follows the aircraft, the unit was never the problem. - **Supplier claim.** Rogue clauses in supply contracts trigger on exactly the pattern the history documents: removals, dates, findings per serial number. A queryable history is the evidence. ## What Good Looks Like A shop with rogue tracking under control knows its repeat offenders by name. Retest alerts surface a handful of serials a month, each of which gets a history review before its next standard test cycle. Units on their third visit go to deep test, not back to stock. And when a supplier conversation happens, the shop brings the serial's full documented trajectory instead of an anecdote. ### Component Test Records for EASA Part-145 URL: https://www.tofupilot.com/guides/component-test-records-for-easa-part-145 Map EASA Part-145 record-keeping and calibration rules to TofuPilot features: test records behind the Form 1, retention, backups, and traceability. An EASA Part-145 component shop releases every unit with an EASA Form 1, and behind every Form 1 sits test evidence the regulation requires you to keep, protect, and produce on demand. Most shops meet the letter of this with paper test sheets and PDF printouts. This guide maps the actual Part-145 clauses to what a structured test-data system covers, so the same records that satisfy the auditor also feed your analytics. ## The Clauses That Touch Test Data | Clause | Requirement | TofuPilot feature | |---|---|---| | 145.A.40(b) | Test equipment controlled and calibrated to an officially recognised standard; calibration records and traceability kept | Station identifier on every run, linking results to the bench and its calibration register | | 145.A.50(d) | CRS for components issued as EASA Form 1, backed by acceptance test evidence | Complete run record per release: measurements, limits, result, date, unit identity | | 145.A.55(a) | Retain all records necessary to prove requirements were met for the CRS | Immutable run history per serial number | | 145.A.55(a)(3) | Detailed maintenance records kept 3 years from CRS issue | Cloud retention beyond the mandated period | | AMC 145.A.55 | Electronic systems: backup within 24 hours of new entries, safeguards against unauthorised alteration, backup stored in a different location | Managed storage with replication; records not editable after upload | Two details from the AMC and guidance material are worth knowing precisely. First, the acceptance test report should be attached to a Form 1 issued as Inspected/Tested, and Block 12 must cite the maintenance data with its revision status; "in accordance with the CMM" alone is not acceptable. Second, the retention clock runs from when the record was created or last amended, so a system that timestamps every run gives you the retention basis for free. ## Structure the Test So the Record Is the Evidence The regulation asks for proof that the unit met its acceptance criteria. A structured test produces that proof as a side effect of running. Each measurement carries its limits, its unit, and a description tying it back to the maintenance data: ```yaml filename="procedure.yaml" name: LRU Bench Acceptance Test version: 1.0.0 description: Functional acceptance test for a returned unit on the repair bench unit: serial_number: default_value: "LRU000001" part_number: default_value: "622-1234-001" main: - name: Power Supply Check python: phases.power_check measurements: - name: supply_voltage unit: V description: "28V nominal bus input, per CMM test 1A" validators: - operator: ">=" expected_value: 27.5 - operator: "<=" expected_value: 28.5 ``` The run record then contains everything the release depends on: the serial number and part number, each measured value against its declared limit, the procedure name and version, the station that ran it, and the timestamp. When the Form 1 goes out as Inspected/Tested, the attached test report is a query, not a scan. Procedure versioning matters here more than it first appears. Block 12 requires the revision status of the maintenance data used. Versioning your test procedure alongside the CMM revision it implements means every historical run states which revision of the test it was measured against. ## Calibration Traceability by Station 145.A.40(b) requires calibration records and traceability for the test equipment itself, and the audit checklist covers tools, calibration, and alternative tooling explicitly. TofuPilot does not replace your calibration register, but the station identifier on every run creates the link auditors look for: name stations to match the calibration register, and any measurement can be traced to the bench that produced it and from there to its calibration state on that date. This also answers the ugly retrospective question. When a tool turns up out of calibration past its due date, the shop must review the work that used it. With runs filtered by station and date range, the affected releases are a query instead of a work-order archaeology project. ## Retention, Backup, and Integrity AMC 145.A.55 sets concrete expectations for computer record systems: a backup updated within 24 hours of any new entry, protection against unauthorised alteration, and backup media held in a different location from the working data. A shop running its records on a spreadsheet and a shared drive owns all three of those problems itself, and the control of computer record systems is a standing item on the EASA audit checklist (MOE chapter 2.21). Structured cloud records shift that burden: runs are immutable once uploaded, storage is replicated, and every record is retrievable by serial number, part number, procedure, station, date range, or result for the full retention period and beyond. For shops working under FAA approval as well, 14 CFR 145.219 asks for records demonstrating compliance for at least 2 years from return to service, in English, in a format acceptable to the FAA; electronic record systems come under AC 120-78B. The EASA 3-year requirement is the stricter of the two, so a system sized for Part-145 covers both. ## What Good Looks Like In a shop running this way, an auditor asking for the test evidence behind a given Form 1 gets it by serial number in seconds, with limits, values, procedure revision, and station attached. The calibration question traces from any measurement to a named bench. Retention is not a filing discipline but a property of the system. And because the compliance records are structured measurements rather than PDFs, the same data answers the engineering questions too: yield by part number, drift by measurement, and the units that keep coming back. ### Build a Reliability Case with Test Data URL: https://www.tofupilot.com/guides/build-a-reliability-case-with-test-data Learn how to turn bench measurements into a reliability case for OEM design changes, using MTBUR gaps, drift trends, and Cpk evidence from TofuPilot. The manufacturer sees a component once, at production. The operator sees it across its whole service life: every removal, every shop visit, every measured value on the bench, year after year. That asymmetry means airlines and MRO shops often understand a part's real failure behavior better than the OEM does. What they usually lack is the evidence in a form that carries weight. This guide shows how to build that case from your own bench data. ## The Data You Hold and the OEM Does Not The industry already runs feedback pipes from operators to manufacturers. ATA Spec 2000 Chapter 11 standardizes the exchange of removal and shop-findings records; airframers publish component performance reports benchmarking MTBUR by part number across reporting fleets; component OEMs run programs that take monthly removal data and return corrective-action feedback. And continued-operational-safety processes depend on information supplied voluntarily by operators, which is the plainest statement of where the trend data originates. But these pipes carry events: removals, findings, hours. They do not carry measurements. The OEM learns that your fleet removed 14 units of a part last year and that 10 tested no fault found. It does not learn that the passing units' output delay has been creeping toward the CMM limit for two years. That second fact is the design-change argument, and only your bench has it. ## MTBUR, MTBF, and the Gap Two numbers frame every component reliability discussion: - **MTBF** counts confirmed failures. - **MTBUR** counts unscheduled removals, whether or not a fault was confirmed. MTBUR is always the lower number, and the gap between them is mostly no-fault-found. The gap is also where the money is: purchase agreements commonly contain a guaranteed MTBUR that the operator can claim against, and rogue-unit clauses that trigger on documented repeat removals per serial number. Both claims stand or fall on the quality of the operator's own records. ## Structure Tests to Produce Evidence A reliability case needs measured values, not verdicts. Each bench test should record the value, the unit, and the limit it was judged against: ```yaml filename="procedure.yaml" main: - name: Signal Integrity python: phases.signal_check measurements: - name: output_delay unit: us validators: - operator: "<=" expected_value: 12.0 - name: signal_snr unit: dB validators: - operator: ">=" expected_value: 40.0 ``` With every shop visit recorded this way, the part number accumulates exactly the dataset a design-change argument is built from. ## Reading the Evidence Four views in TofuPilot turn the accumulated runs into the case: - **Distribution against limits.** If the fleet's values for a measurement crowd one limit rather than centering, the design margin is thinner in service than the spec assumed. A histogram of two years of inductions makes that visible in one image. - **Cpk.** A Cpk below 1.0 on a CMM limit says the population does not reliably fit its own acceptance criteria. That is a number an OEM reliability engineer cannot wave away, because it is computed on their limits. - **Drift over time.** A mean shifting across months of shop visits separates aging from randomness, and puts an onset date on the change. Onset dates correlate with mod status, supplier changes, and batch boundaries. - **Per-serial history.** The repeat offenders, documented visit by visit. This is the direct input to guaranteed-MTBUR and rogue-unit claims, and the concrete illustration that makes the statistical case tangible. ## Making the Case A design-change proposal built this way has a standard shape: the removal and NFF history (which the OEM partly knows), plus the measurement story (which it does not). For example: this part number shows MTBUR at half the guaranteed value; 60 percent of removals test NFF on the bench; but across 200 inductions the output delay distribution has shifted 20 percent toward the CMM limit, with onset eighteen months ago; the serials past a threshold of shop visits share the same drift signature. That is no longer "our mechanics do not trust this box". It is a dataset the OEM's own reliability process can act on: a modification, a CMM limit revision, or a warranty settlement. Export the runs behind each figure and the case ships with its evidence attached. ## What Good Looks Like An operator working this way stops being a passive reporter of removals and becomes the party at the table with the best data. Reliability reviews with the OEM start from the operator's distributions rather than the OEM's summaries. Warranty and guaranteed-MTBUR claims are filed with measurement evidence, not just removal counts. And when the OEM disputes a trend, the answer is the query, rerun in front of them. ### Reduce No-Fault-Found (NFF) in Testing URL: https://www.tofupilot.com/guides/reduce-no-fault-found-nff-in-testing Learn what No-Fault-Found means, why units pass on the bench yet fail in service, and how to catch the pattern using your own test data in TofuPilot. No-Fault-Found is a returned unit that tests good on the bench. Nothing reproduces, so the unit ships back into service, often to fail again weeks later. Across component maintenance, NFF accounts for a large share of returns, and every one of them burns bench time while eroding trust in the test itself. This guide explains what NFF is, why it happens, and how to catch the pattern behind it using your own test data. ## What No-Fault-Found Means A unit comes off equipment reported as faulty. On the bench it passes every check. With no failure to act on, the unit is returned to service as serviceable. That is an NFF, sometimes logged as CND (Cannot Duplicate) or RTOK (Retest OK). NFF is not one problem. It is a label for several: - **Intermittent faults.** The defect is real but only appears under conditions the bench does not reproduce (temperature, vibration, a specific input sequence). - **Test escape.** The bench limits are too loose, so a marginal unit passes. - **Wrong removal.** The fault was elsewhere in the system, and a good unit was pulled. - **Fixture drift.** The bench itself has changed, so a real fault reads as a pass. You cannot fix what you cannot see. The first step is making each of these visible in the data, which starts with how the test is written. ## Structure the Test So NFF Is Visible An NFF investigation needs three things in the record: the exact measured value (not just pass/fail), the serial number, and a stable set of limits to trend against. A test that only reports pass or fail throws away the information you need. The example below is a bench acceptance test for a returned unit, written with the TofuPilot framework. Each phase reads a real value and validates it against a limit, so a passing unit still leaves a full measurement trail behind. ```yaml filename="procedure.yaml" name: LRU Bench Acceptance Test version: 1.0.0 description: Functional acceptance test for a returned unit on the repair bench unit: auto_identify: true serial_number: default_value: "LRU000001" part_number: default_value: "622-1234-001" plugs: - name: bench python: plugs.bench:Bench main: - name: Power Supply Check python: phases.power_check measurements: - name: supply_voltage unit: V validators: - operator: ">=" expected_value: 27.5 - operator: "<=" expected_value: 28.5 - name: Signal Integrity python: phases.signal_check measurements: - name: output_delay unit: us validators: - operator: "<=" expected_value: 12.0 - name: signal_snr unit: dB validators: - operator: ">=" expected_value: 40.0 ``` ```python filename="phases/power_check.py" def power_check(measurements, bench): measurements.supply_voltage = bench.read_supply_voltage() ``` ```python filename="phases/signal_check.py" def signal_check(measurements, bench): measurements.output_delay = bench.read_output_delay() measurements.signal_snr = bench.read_snr() ``` ```python filename="plugs/bench.py" class Bench: def read_supply_voltage(self) -> float: return 28.01 def read_output_delay(self) -> float: return 9.4 def read_snr(self) -> float: return 44.7 ``` Run it from the procedure directory: ```bash tofupilot run ./lru-bench-test ``` ```text → Phase: Power Supply Check → Phase: Signal Integrity ✓ Phase Power Supply Check: PASS → supply_voltage = 28.01 V [PASS] ✓ Phase Signal Integrity: PASS → output_delay = 9.4 us [PASS] → signal_snr = 44.7 dB [PASS] ✓ Run complete: PASS ``` This unit passes. That is the NFF case. But because the test recorded `output_delay = 9.4 us` against a 12.0 limit, not just PASS, the next unit that reads 11.8 is visible as a marginal pass long before it fails in service. The value is what lets you trend. The pass/fail alone tells you nothing. ## Track Retest Counts The strongest NFF signal is a unit that gets tested more than once. A single unit cycling through the bench three or four times is either intermittent or being chased in circles. Because each run identifies the unit by serial number, TofuPilot builds a per-unit history automatically. The unit history page shows every run for a serial number, so a repeat visitor stands out without you tracking it by hand. Sort a part number by retest count and the intermittent units surface at the top. These are the ones worth pulling for a deeper look, and the ones a supplier discussion should be built around. ## Trend the Measurement, Not the Verdict A unit that passes at the edge of its limit is an NFF waiting to happen. To catch it, look at the measured value over time rather than the pass rate. In TofuPilot, open the measurement for the part number and read the distribution and the drift: - **Distribution.** If passing values cluster near a limit instead of centering in the range, the design or the process is running close to the edge. Marginal units are passing today and will fail tomorrow. - **Cpk.** A Cpk below 1.0 means the spread does not fit inside the limits reliably. Some fraction of units will always fall out, and some of those are your NFF returns. - **Drift.** A slow shift in the mean points to aging, a supplier change, or a bench that has moved. Catching the shift early is the difference between one investigation and a batch of returns. This is the same statistical basis manufacturing teams use for yield, applied to a repair line. The question shifts from "did this unit pass" to "is this measurement healthy across every unit we see." ## Alert on the Signal Automatically Reviewing charts by hand does not scale. TofuPilot can watch the data and raise an alert when the NFF pattern appears: - **Measurement drift.** Fires when a measurement moves off its own baseline, whether a mean shift or a falling Cpk, so a drifting part is flagged before it becomes a wave of returns. - **Retest threshold.** Fires when a unit is tested more often than the procedure's normal rate. The threshold self-calibrates per procedure, so you are alerted on the genuinely abnormal units, not the routine ones. The alerts learn each procedure's normal behavior, so you set them once rather than hand-tuning limits per part. ## What Good Looks Like A repair line with NFF under control has a few visible traits. Retest counts are low and flat, with few units cycling back. Measurement distributions center inside their limits rather than crowding an edge, and Cpk stays above 1.0. When a value does start to drift, an alert catches it while it is still one part, not a batch. None of this requires new hardware or access to anyone's proprietary software. It comes from recording the measured value instead of the verdict, keeping a per-unit history, and trending what you already capture on the bench. ### Avionics LRU Bench Testing and Analytics URL: https://www.tofupilot.com/guides/avionics-lru-bench-testing-and-analytics Learn how avionics LRU bench tests work, from induction to release, and how to turn CMM test results into analytics with TofuPilot. An avionics LRU (line-replaceable unit) that comes off an aircraft passes through a well-defined shop flow: induction, functional test, fault isolation, repair, final acceptance, release. Every step produces measurements. In most shops those measurements end their life as an ATE printout or a PDF attached to the release certificate, which means the shop learns nothing from them beyond the single pass/fail verdict. This guide walks through the LRU bench workflow and shows how to capture the same test as structured data you can trend. ## The LRU Shop Workflow A removed unit arrives with its removal paperwork and goes through: 1. **Induction.** Receiving inspection, identity check against the removal record. 2. **Functional test.** Run per the OEM's CMM (component maintenance manual), which defines the test conditions, limits, and maintenance levels for the unit. 3. **Fault isolation.** If the functional test fails, isolate to the board or subassembly. 4. **Repair.** Replace or rework the faulty element. 5. **Final acceptance test.** Full CMM test again, from scratch, on the repaired unit. 6. **Release.** Certificate of release to service backed by the acceptance test evidence. Note the loop hiding in step 2: a significant share of inducted avionics units pass the functional test with no fault found. Published figures put NFF at 20 to 50 percent of avionics shop inductions. Those units consume a full bench cycle and return to stock with nothing learned, unless the measurements are kept. ## What the Bench Measures A typical avionics LRU acceptance test covers a few functional areas, each with its own measurements and limits from the CMM: - **Power.** Input current, supply voltages, inrush behavior. - **Bus communication.** ARINC 429 word error rates, response times, BITE message readout. - **Signal and calibration.** Output accuracy against reference, delays, signal-to-noise. - **Environmental screening.** For intermittent suspects, temperature and vibration during test, informally known as bake and shake. ## Write the Acceptance Test as Structured Data The test below is a final acceptance test for a repaired LRU, written with the TofuPilot framework. The procedure file declares the phases and limits; each phase is a plain Python function reading from the bench. ```yaml filename="procedure.yaml" name: LRU Final Acceptance Test version: 1.0.0 description: Final acceptance test for a repaired avionics LRU before release unit: auto_identify: true serial_number: default_value: "LRU000042" part_number: default_value: "622-1234-001" plugs: - name: bench python: plugs.bench:Bench main: - name: Power Consumption python: phases.power measurements: - name: input_current unit: A validators: - operator: "<=" expected_value: 1.8 - name: Bus Communication python: phases.bus_comms measurements: - name: arinc429_word_error_rate validators: - operator: "<=" expected_value: 0.00001 - name: response_time unit: ms validators: - operator: "<=" expected_value: 50 - name: Output Calibration python: phases.calibration measurements: - name: output_accuracy_error unit: "%" validators: - operator: ">=" expected_value: -0.5 - operator: "<=" expected_value: 0.5 ``` ```python filename="phases/power.py" def power(measurements, bench): measurements.input_current = bench.read_input_current() ``` ```python filename="phases/bus_comms.py" def bus_comms(measurements, bench): measurements.arinc429_word_error_rate = bench.read_word_error_rate() measurements.response_time = bench.read_response_time() ``` ```python filename="phases/calibration.py" def calibration(measurements, bench): measurements.output_accuracy_error = bench.read_accuracy_error() ``` ```python filename="plugs/bench.py" class Bench: def read_input_current(self) -> float: return 1.42 def read_word_error_rate(self) -> float: return 0.0 def read_response_time(self) -> float: return 31.5 def read_accuracy_error(self) -> float: return 0.12 ``` Run it from the procedure directory: ```bash tofupilot run ./lru-acceptance-test ``` ```text → Phase: Power Consumption → Phase: Bus Communication → Phase: Output Calibration ✓ Phase Output Calibration: PASS → output_accuracy_error = 0.12 % [PASS] ✓ Phase Bus Communication: PASS → arinc429_word_error_rate = 0.0 [PASS] → response_time = 31.5 ms [PASS] ✓ Phase Power Consumption: PASS → input_current = 1.42 A [PASS] ✓ Run complete: PASS ``` The plug in this example returns fixed values so the test runs anywhere; on a real bench the same functions read from your instruments over VISA, serial, or whatever the fixture speaks. The structure is the point: every value is recorded with its unit and limit, tied to the serial number and part number. ## From Test Records to Analytics Once acceptance tests flow in as structured runs, the analytics come for free: - **Per part number.** First-pass yield across all units of a part, the failure Pareto showing which phase and measurement fail most, and Cpk on each measurement showing how much margin the fleet of units really has against the CMM limits. - **Per serial number.** The unit history page lists every shop visit and every measured value for one serial. A unit inducted for the fourth time is visible at a glance, along with how its measurements moved between visits. - **Over time.** Drift in a measurement across months of inductions points at aging in the field, a supplier change in a repaired subassembly, or the bench itself moving. None of this requires the OEM's software or data. The measurements are produced by your own bench under your own test procedure; recording them structurally instead of printing them is the only change. ## What Good Looks Like A component shop running this way releases every unit with the same certificate as before, but keeps the evidence behind it queryable. The engineer asked "have we seen this failure before" answers from the part number's history instead of a filing cabinet. The units that keep coming back stand out by retest count. And when a part's measurements start crowding a CMM limit across many serials, the shop sees it while it is still a trend, not yet a backlog. ### Getting Started with TofuPilot for Aviation MRO URL: https://www.tofupilot.com/guides/getting-started-with-tofupilot-for-aviation-mro Set up the TofuPilot CLI, write a first component bench test, and run it locally in an aviation MRO shop, with no account or proprietary data needed. Component maintenance shops build their test benches in-house, and the software side is usually a mix of instrument-control scripts, spreadsheets, and printouts. TofuPilot adds a structured layer on top of the scripts you already write: tests defined in a YAML procedure, phases in plain Python, and every measurement recorded with its limits and unit identity. This tutorial takes you from nothing to a running component bench test, entirely locally. No account is needed to start, and nothing here touches OEM software or proprietary data; the framework only handles measurements your own bench produces. ## Prerequisites - A machine running Windows, Linux, or macOS - Python installed (the CLI can provision a virtual environment for you) ## Step 1: Install the CLI The TofuPilot CLI is a single executable. The framework and CLI are both open source under the MIT license. On Linux or macOS: ```bash curl -fsSL https://tofupilot.sh/install | sh ``` On Windows (PowerShell): ```powershell $p = "$env:TEMP\tp-install.ps1"; irm https://tofupilot.sh/install.ps1 -OutFile $p; powershell -ExecutionPolicy Bypass -File $p; ri $p -EA 0 ``` ## Step 2: Create the Procedure A procedure is a directory with a `procedure.yaml` declaring the test sequence, plus the Python it references. Create this layout: ```text lru-bench-test/ procedure.yaml phases/ power_check.py plugs/ bench.py ``` The YAML declares one phase with one validated measurement, and identifies the unit under test by serial number and part number, which is what ties every result to a unit's history: ```yaml filename="procedure.yaml" name: LRU Bench Acceptance Test version: 1.0.0 description: First bench test for a returned component unit: auto_identify: true serial_number: default_value: "LRU000001" part_number: default_value: "622-1234-001" plugs: - name: bench python: plugs.bench:Bench main: - name: Power Supply Check python: phases.power_check measurements: - name: supply_voltage unit: V validators: - operator: ">=" expected_value: 27.5 - operator: "<=" expected_value: 28.5 ``` ## Step 3: Write the Phase and the Plug The phase is a plain Python function. It receives the measurements object and any plugs it needs by name: ```python filename="phases/power_check.py" def power_check(measurements, bench): measurements.supply_voltage = bench.read_supply_voltage() ``` The plug is a class wrapping your instrument. This one returns a fixed value so the example runs anywhere; on a real bench the same method would read from your power supply over VISA or serial: ```python filename="plugs/bench.py" class Bench: def read_supply_voltage(self) -> float: return 28.01 ``` This split is the pattern for everything that follows: procedures declare what is measured and what its limits are, phases contain the test logic, plugs contain the instrument drivers. Your existing instrument-control code moves into plugs mostly unchanged. ## Step 4: Run It From the parent directory: ```bash tofupilot run ./lru-bench-test ``` The CLI resolves a Python environment (offering to create one if missing), executes the phases, evaluates each measurement against its validators, and prints the result: ```text → Phase: Power Supply Check ✓ Phase Power Supply Check: PASS → supply_voltage = 28.01 V [PASS] ✓ Run complete: PASS ``` The run executed entirely on your machine, without contacting anything. ## Step 5: Connect the Dashboard When you want history and analytics on top of local runs, connect the procedure to a dashboard. Runs uploaded from then on build the part number's yield and measurement trends and each serial number's visit history. ```bash tofupilot login tofupilot link ./lru-bench-test tofupilot run ./lru-bench-test --upload ``` Low-volume use fits the free tier, which suits component maintenance well: a shop seeing a handful of units per part number per month generates far fewer runs than a production line, while each run carries more history value. From here, grow the procedure the way your CMM test grows: more phases for more functional areas, more measurements per phase, plugs for each instrument on the bench. The measurements accumulate into the part number's record, which is where the analytics for no-fault-found, drift, and repeat visitors come from. ### How to Detect Measurement Drift with TofuPilot URL: https://www.tofupilot.com/guides/how-to-detect-measurement-drift-with-tofupilot Learn how TofuPilot grades measurement drift against each series' own baseline, and how to write tests that make a real shift visible early. Measurement drift is a gradual shift in your test data over time. Parts still pass today, but the distribution is creeping toward a spec limit. TofuPilot grades every numeric measurement series for drift automatically, so the question is no longer whether someone remembered to look at the chart. This guide explains what triggers a drift alert, and how to write tests that make a real shift visible early. ## What Causes Drift Drift has real physical causes. Knowing them helps you investigate when an alert fires. Instrument aging is the most common. A DMM's calibration drifts over months. A current sense resistor changes value with thermal cycling. A force gauge spring weakens with use. Fixture wear matters too. Pogo pins lose spring force after thousands of contacts. Test sockets develop intermittent connections. Alignment features wear down, changing DUT positioning. Environmental changes are subtler. Seasonal temperature shifts affect analog measurements. Humidity changes impact high-impedance readings. Facility voltage fluctuations add noise. Component lot variation comes from your supply chain. A new reel of resistors from a different lot shifts your circuit's behavior slightly. The parts are all in spec, but the distribution center moved. ## How TofuPilot Detects Drift There is no threshold to configure. Each measurement series is judged against **its own history**, not against an absolute limit. TofuPilot maintains a smoothed average of the series and replays that same statistic at many points across the measurement's past. That replay answers the only question that matters: how much does this measurement normally move? The current gap between the smoothed average and that baseline is then expressed in **σ (sigma)** — multiples of the series' own normal variation. | Severity | Fires at | |----------|----------| | Info | ≥ 3σ | | Warning | ≥ 4σ | | Critical | ≥ 5σ | Because the scale is relative, a 2 mV shift on a rock-steady reference and a 200 mV shift on a noisy rail can both be the same severity. That is the point: a threshold that is right for one measurement is wrong for the next one. Two properties are worth knowing when you read an alert: - **Each part is its own series.** Two product variants tested by one procedure are two physical populations, so they never share a baseline. The alert names the part it fired on. - **Stations are pooled.** Two benches running the same procedure share one baseline, which maximizes coverage but dilutes a problem confined to a single bench. If one station's volume justifies its own baseline, scope a custom rule to it. A series needs at least 70 values before it is graded at all, and 100 before the full severity ladder unlocks. If a new line produces no drift alerts, this is usually why. The full method, including the noise guards that deliberately suppress alerts, is documented in [automatic detection](/docs/alerts/automatic-detection). ## One Alert per Event, Not per Measurement A single physical cause rarely moves a single measurement. A degrading fixture or a warm afternoon moves everything downstream of it at once. TofuPilot groups automatic drift into one alert per procedure and part. Every drifting measurement attaches to that alert with its own entry in the timeline, and the alert names the strongest one. It resolves only once every attached measurement has recovered. So a bad night on one line reads as one incident with six measurements listed inside it, not six separate notifications. ## Writing Tests That Reveal Drift Drift detection needs consistent, repeatable measurements with enough resolution to see small shifts. Use physical units, set limits with margin, and avoid rounding. Rounding deserves emphasis: a value quantized to a coarse grid carries no information below one step of that grid, and TofuPilot will not report a shift finer than the step it detects in your data. Recording 2.4988 V rather than 2.5 V is what makes early drift visible at all. ```python filename="drift_sensitive_test.py" import openhtf as htf from openhtf.util import units from tofupilot.openhtf import TofuPilot @htf.measures( htf.Measurement("ref_voltage") .with_units(units.VOLT) .in_range(minimum=2.495, maximum=2.505), htf.Measurement("temp_sensor") .in_range(minimum=23.0, maximum=27.0), htf.Measurement("adc_offset") .in_range(minimum=-3, maximum=3), ) def calibration_check(test): """Measure reference points that are sensitive to drift.""" test.measurements.ref_voltage = 2.4988 test.measurements.temp_sensor = 24.6 test.measurements.adc_offset = 1 @htf.measures( htf.Measurement("contact_resistance") .in_range(maximum=0.100) .with_units(units.OHM), htf.Measurement("leakage_current") .in_range(maximum=0.000001) .with_units(units.AMPERE), ) def fixture_health_check(test): """Track fixture-related measurements that degrade over time.""" test.measurements.contact_resistance = 0.0423 test.measurements.leakage_current = 0.00000012 @htf.measures( htf.Measurement("gain") .in_range(minimum=19.5, maximum=20.5), htf.Measurement("phase_margin") .in_range(minimum=45.0), htf.Measurement("output_impedance") .in_range(maximum=2.0) .with_units(units.OHM), ) def analog_performance_test(test): """Measure analog parameters prone to component lot variation.""" test.measurements.gain = 20.05 test.measurements.phase_margin = 52.3 test.measurements.output_impedance = 1.1 def main(): test = htf.Test( calibration_check, fixture_health_check, analog_performance_test, ) with TofuPilot(test): test.execute(test_start=lambda: "UNIT-1001") if __name__ == "__main__": main() ``` The `fixture_health_check` phase is particularly useful. Contact resistance that climbs from 40 to 80 mohm over a few weeks tells you the fixture needs maintenance, even though every reading still passes. Splitting measurements by channel rather than collapsing them into one pass/fail is the other habit that pays. A drift confined to channel 3 is actionable; a generic "the LED test is failing more" is not. ## Investigating a Drift Alert The alert tells you a series moved. The measurement data tells you why. Open the series in TofuPilot and read the trend chart first: drift shows up as a consistent slope. The control chart adds the statistical view, flagging the classic Western Electric patterns — one point beyond 3σ, two of three consecutive points beyond 2σ on the same side, four of five beyond 1σ on the same side, and eight consecutive points on one side of the center line. Then narrow the scope before you touch any hardware. Filter by station and by time range. Because stations share one baseline, a drift caused by a single worn fixture shows up as a pooled shift, and the station filter is what separates "the process moved" from "bench 2 moved". ## Responding to Drift When you confirm drift, the response depends on the cause. For instrument drift, recalibrate and verify with a known reference, then check whether your calibration interval is too long. For fixture wear, inspect and replace the worn components. Track the fixture's cycle count and set preventive maintenance from the drift data you have collected. For environmental drift, correlate with facility logs. If temperature is the driver, improve environmental control around the station or add temperature compensation to the measurement. For component lot variation, compare the distributions before and after the lot change. If the shift is significant but still in spec, the new baseline will establish itself as the series accumulates values. If it is borderline, work with your supplier on tighter incoming specs. Once the cause is fixed, the alert closes on its own: the series has to measure recovered on three consecutive values before TofuPilot resolves it, which is what stops an alert from closing on a single lucky reading. ### Setting Alert Thresholds That Actually Work URL: https://www.tofupilot.com/guides/setting-alert-thresholds-that-actually-work Why a fixed yield threshold fails on real production lines, and how a self-calibrating EWMA detector finds the right band for each one. Your line tests units all day and you track its first-pass yield (FPY). At what point should someone be alerted? The intuitive answer — pick a threshold — runs into two walls immediately. The first is level. A procedure running at 99% FPY should alert at 98%, and loudly at 97%: every point lost there is enormous. A procedure running at 50% — a deliberately harsh burn-in, say — should only alert around 45%, and loudly at 40%. No absolute threshold fits both. The second is intrinsic variability. Two lines can share the same average FPY of 55%: one is very stable and stays within ±1 point; the other naturally swings between 50% and 60% depending on lots and shifts. A drop to 48% is a major event on the first, an ordinary Tuesday on the second. The relevant threshold therefore depends not only on the level, but on each line's habitual behaviour. A trustworthy alerting system has to calibrate itself on both axes, for every procedure, without asking the user to guess constants. Here is how statistics goes about it, and the choices we drew from it. ## The state of the art: detecting a change in a stream of tests The problem has had a name for a century: **statistical process control** (SPC), founded by Walter Shewhart in the 1920s. Its central idea fits in one sentence: every process has routine variation (its "common causes"), and you should only alert when an observation leaves that routine (a "special cause"). Shewhart charts judge each point in isolation; two more recent methods, still the current references, add the decisive ingredient for small drifts: memory. ### CUSUM: the evidence accumulator The **CUSUM** chart (cumulative sum, Page, 1954) processes each tested unit, one by one, and maintains a counter of evidence. An electronics analogy: an integrator with a constant leak and a floor at zero. Every failure injects charge, every pass lets a little leak away; on a healthy process the leak wins and the counter sits at zero; after a genuine drift the injections take over and the counter climbs inexorably to the alarm threshold. The version adapted to pass/fail streams under 100% inspection (the Bernoulli CUSUM, Reynolds & Stoumbos, 1999) is explicitly presented by its authors as superior to the practice of grouping units into samples or time windows. And CUSUM holds a rare title in statistics: for a given false-alarm rate, it is provably the fastest detector for the drift it was specified for (Moustakides, 1986). ### EWMA: smoothed yield The **EWMA** chart (exponentially weighted moving average, Roberts, 1959) is a low-pass filter applied to the pass/fail pulse train: its output is a smoothed estimate of the current failure rate, where each unit weighs slightly less than the next one — old data fades gently instead of falling off a cliff as it does in a sliding window. A single parameter, λ, sets the memory depth, expressed in units produced rather than clock time: a fast line covers its memory in a few hours, a slow line in a few days, and both are monitored with the same statistical rigour. You alert when the smoothed yield departs too far from its reference value, the gap being measured in standard deviations. The difference in nature matters for what follows: EWMA is a **gauge** (its output reads in the unit of the metric — "smoothed FPY: 91.2%"), CUSUM is an **accumulator** (its output, "2.9 units of evidence toward a limit of 4", has no direct interpretation for a human eye). ### What about the other methods? Others exist — self-sizing windows from data stream mining (ADWIN), Bayesian changepoint detection, adaptive sampling charts — which we will not explore here: they are either designed for other data regimes, or considerably harder to explain and audit for a quality engineer, which matters when an alert is supposed to trigger action on a production floor. References are at the end of the article for the curious. ## The textbook trap: one noise model for every line Open an SPC textbook and the formulas for CUSUM as well as EWMA on pass/fail data rest on the same assumption: the noise is binomial. In other words, every unit would have exactly the same probability of failure, and a line's variability would follow entirely from its average level through a formula, without ever being measured. Real production does not honour that contract. The true failure probability wanders from hour to hour: shift changes, component lots, temperature, tooling wear. As a result the observed variability exceeds — sometimes by a lot — what the formula predicts. The phenomenon has a name, **overdispersion**, and it is the best-documented defect of attribute charts applied as-is in a factory (it is the problem Laney's p′ chart, 2002, was invented to correct). During our training phase on real production datasets (more on that below), we measured variability ranging from 1× to more than 5× the binomial prediction depending on the line, on procedures that were all perfectly healthy. The consequences of a textbook setting are then inescapable. Calibrate the thresholds for the fluctuating line and they go blind on the stable one: real drops slip under the radar. Calibrate them for the stable line and the fluctuating one fires false alerts in bursts — and everyone knows what follows, documented in every factory in the world: the alerts end up ignored, including the true ones. This is precisely why "picking a threshold" is so hard: there is no good global threshold, only a good threshold per line, and it changes when the line changes. ## Why EWMA answers this better Because the EWMA filter's output is a gauge in the unit of the metric, one correction suggests itself naturally: instead of computing its dispersion theoretically from the binomial formula, **measure it**. Replay the filter over the line's own history, observe the real distribution of its values, and the alert band becomes: "the smoothed yield departs by N observed standard deviations from its own historical behaviour". Both axes of the problem are then covered at once: - **Level.** The reference is estimated per procedure, and the band's natural width follows the level. The noise of a pass/fail stream is √(p(1−p)): it is mechanically finer at 99% FPY than at 50%. The excellent line is monitored to the point, the difficult line to the multiple of points. - **Intrinsic variability.** The line that sits steady at 55% has a tight smoothed history, so a narrow band; the line swinging between 50% and 60% has a wandering smoothed history, so a wide one. Each is judged against itself. Can the same correction be applied to CUSUM? Only with difficulty, and this is the technical point that separates the two methods: CUSUM's entire calibration — the alarm threshold, the optimality guarantee, the advertised false-alarm rate — is welded to the binomial model by the very construction of its evidence counter. Under overdispersion those guarantees evaporate, and the variants that restore them require fitting an additional statistical model per line and recalibrating by simulation. Workable in a study, not in a product monitoring hundreds of heterogeneous procedures without human intervention. Then there is the argument we care about most: what the user sees. An EWMA alert **shows itself**. The smoothed yield curve is plotted in the unit the quality engineer knows, with its band of normality around it, and the alert reads off it: "smoothed FPY at 91.2% against a normal of 95.1% ± 1.2". Every number is recomputable by hand from the runs. A CUSUM alert has to be translated: the evidence counter has no meaningful axis, and the counts that explain it must be reconstructed after the fact. For a system whose goal is trust — that a quality engineer can audit why they were woken up — the gauge beats the accumulator. ## What is left to fix, and why we trained it on real data Self-calibration does not remove every constant; it changes their nature. Everything that depends on the line (reference level, variability) is measured continuously. Two universal settings remain, the same for everyone: - **λ, the memory depth.** In reality a choice of boundary between what counts as weather (fast fluctuations, absorbed into the band) and what counts as change (sustained drifts, which alert). It is chosen through a promise about reaction time: "confirm a sustained drift within one to two hundred units". - **The severity scale.** The standard-deviation multipliers separating info, warning and critical. This is a false-alarm budget: at what historical rarity does something deserve to disturb someone? How do you set these values without falling back into arbitrariness? By training them: replay the candidate detector, setting by setting, over real production datasets — tens of thousands of tests covering widely varied profiles (lines at 99% as well as 50% FPY, production in bursts of several hundred units a day as well as steady lines running a few units a day). Then count: how many alerts each setting would have raised, at what severity, and whether the confirmed incidents would have been detected, and within how many units. That training settled three things. First λ = 0.02 (a useful memory of roughly 50 to 150 units): shorter settings widen the band faster than they deepen the excursions and lose the critical signal on real incidents; longer ones double the reaction time with no detection gain. Next the 2σ/3σ/5σ scale: over two months of real data, 2σ corresponds to a handful of informative episodes per line, 3σ almost exclusively to genuine events, and 5σ exclusively to one authentic incident (an FPY that fell from 97% to 80% over about a hundred units, classified critical by the detector). Finally a guard rail: the measured band is floored by the binomial minimum, so that a flawless line (100% FPY over thousands of units, zero observed dispersion) cannot turn its first failure into an "infinite" alarm. One last check, from outside the data: we gave sample datasets to an experienced quality engineer and asked at which levels they would have wanted to be alerted. With λ = 0.02, the computed band lands on their answers: at 99% FPY, alert at 98% and strong alert at 97%; at 50%, alert at 45% and strong at 40%. A calibration derived from replay and a field intuition converging without having consulted each other is the kind of agreement you look for. And the training is not a one-off event: the replay harness is kept, and every evolution of the detector goes through it again before being deployed. ## Sources and further reading Main methods: - E. S. Page, *Continuous Inspection Schemes*, **Biometrika** 41(1-2), 1954: the founding paper of CUSUM. - S. W. Roberts, *Control Chart Tests Based on Geometric Moving Averages*, **Technometrics** 1(3), 1959: the founding paper of EWMA. - G. V. Moustakides, *Optimal Stopping Times for Detecting Changes in Distributions*, **Annals of Statistics** 14(4), 1986: the CUSUM optimality proof (open access). - M. R. Reynolds Jr. & Z. G. Stoumbos, *A CUSUM Chart for Monitoring a Proportion When Inspecting Continuously*, **Journal of Quality Technology** 31(1), 1999: the Bernoulli CUSUM for pass/fail streams. - D. B. Laney, *Improved Control Charts for Attributes*, **Quality Engineering** 14(4), 2002: overdispersion and the p′ chart. - NIST/SEMATECH, *e-Handbook of Statistical Methods*, EWMA Control Charts section: the practical online reference. - D. C. Montgomery, *Introduction to Statistical Quality Control*, Wiley: the field's reference work (rational subgroups, attribute charts, CUSUM/EWMA). - W. A. Shewhart, *Economic Control of Quality of Manufactured Product*, 1931: the founding work of SPC. The other methods mentioned: - A. Bifet & R. Gavaldà, *Learning from Time-Changing Data with Adaptive Windowing*, **SIAM International Conference on Data Mining**, 2007: self-sizing windows (ADWIN). - R. P. Adams & D. J. C. MacKay, *Bayesian Online Changepoint Detection*, arXiv, 2007: Bayesian changepoint detection. - *A Variable Sampling Interval Synthetic X̄ Chart for the Process Mean*, **PLoS ONE** 10(5), 2015: an example of an adaptive sampling chart. ### Record Test Conditions as Measurements URL: https://www.tofupilot.com/guides/record-test-conditions-as-measurements Learn how to capture supply voltages, temperatures, and other test conditions as measurements so they chart, export, and correlate. A common question: can metadata be attached to an individual measurement? For example, Signal A measured 1 V, and the power supply was at 28 V / 10 mA at that moment. There is no metadata field on individual measurements, and for capture conditions that is the wrong tool anyway. The supply voltage at the moment of capture is itself a measurement. Record it as one, and it becomes first-class data: charted in measurement control, filterable, exportable, and available to correlation analysis. An annotation would be none of those things. The rule of thumb: | The value... | Record it as | |---|---| | Can vary during or between tests (supply voltage, ambient temperature, load current) | A measurement in the same phase | | Is constant for the whole run and not measurable (batch number, fixture id, operator shift) | Run metadata | | Is free-form documentation | A docstring | ## Step 1: Measure the conditions alongside the signal Record the conditions in the same phase as the signal they contextualize: ```python filename="phases/signal_check.py" def signal_check(measurements, supply, dmm): # Conditions first: recorded as measurements, not annotations measurements.supply_voltage = supply.measured_voltage() # V measurements.supply_current = supply.measured_current() # mA # The signal itself measurements.signal_a = dmm.measure_dc_volts("A") ``` Conditions can carry validators too. A supply reading outside its expected window is itself a test result: it tells you the measurement was taken under the wrong conditions, before anyone spends an afternoon debugging the unit. ## Step 2: Use run metadata for run-constant context For values that are constant across the whole run, attach run metadata as key/value pairs at upload. Typical keys: batch, fixture, bench id, firmware under test. Run metadata is filterable in run lists and through the API, but it is per run, not per measurement. ## Step 3: Correlate conditions with results Because conditions are measurements, they participate in the same analytics as everything else. If signal A drifts and supply voltage drifts with it, both series show it in measurement control over the same time axis. If a value deviation only appears under certain conditions, the conditions are in the data, so the relationship can actually be checked instead of remembered. The general principle: correlations can only be computed across things recorded as data. Every condition the bench records as a measurement is one more thing analytics can check without anyone having to know in advance to look. ### Add Derived Measurements to a Test Script URL: https://www.tofupilot.com/guides/add-derived-measurements-to-a-test-script Learn how to record computed values like spreads, margins, and ratios as first-class measurements, so analytics can chart and correlate them. The best early failure indicators are often not measured directly by any instrument: they are computed from values the bench already has. A derived measurement is one of these computed values recorded as a normal measurement, with its own name, unit, and limits. Once recorded, a derived measurement behaves like any other: it appears in measurement control, gets a control chart and Cpk, can carry validators, exports to CSV, and participates in drift alerts. A value that only exists inside a spreadsheet formula does none of that. ## Prerequisites - A test procedure that records at least two related measurements in one phase ## Step 1: Compute the value in the phase Derive the value right where the raw measurements are taken, and record it in the same phase: ```python filename="phases/contact_voltage_drop.py" drops = {} for contact in ["A1", "A2", "A3"]: mv = dmm.measure_dc_millivolts(contact) drops[contact] = mv measurements.contact_drop_a1 = drops["A1"] measurements.contact_drop_a2 = drops["A2"] measurements.contact_drop_a3 = drops["A3"] # Derived: imbalance between poles, a better early indicator # than any single pole against its own limit. measurements.contact_drop_spread = round(max(drops.values()) - min(drops.values()), 2) ``` ## Step 2: Give it a limit A derived measurement earns its keep when it carries its own validator. In the example, each pole has a 100 mV limit, and the spread gets a 40 mV limit. A unit can pass all three pole limits and still fail on spread, which is exactly the class of defect the individual limits cannot see. Set the limit from fleet data, not intuition: record the measurement without a validator first, look at the healthy distribution in measurement control, and place the limit outside it. ## Step 3: Pick derivations that expose failure modes Useful patterns: | Derivation | Computation | What it exposes | |---|---|---| | Spread | max - min across channels | One degrading channel among nominally identical ones | | Margin | limit - measured value | Erosion of headroom before any limit fails | | Ratio | value A / value B | Shifts that scale both values but change their relationship | | Delta from reference | value - golden sample value | Fixture or instrument drift | | Symmetry | left - right, or phase-to-phase | Mechanical or winding asymmetry | ## Step 4: Analyze at fleet level Open measurement control and select the derived measurement over the full history. A healthy fleet is one population; a defect mode shows up as a second one: ![Fleet control chart of a derived spread measurement: the healthy population sits near 11 mV, a second population sits above the 40 mV limit](https://bfusqmdwknceg6ih.public.blob.vercel-storage.com/latent-defects-fleet-spread.jpg-370aab47-3f41-48ce-ad6b-782efdbf5db7-eE3PxVhhRF43WoLy58I9cJqVWWQpLT.jpg) From there the standard tools apply: control chart, histogram, Cpk, per-serial scoping, and CSV export for anything you want to take further. ### Import Historical Test Data via the API URL: https://www.tofupilot.com/guides/import-historical-test-data-via-the-api Learn how to backfill years of legacy test results with the runs API, keeping original test dates so trends and analytics stay correct. Switching test data platforms should not mean starting the fleet history from zero. Trend charts, per-serial degradation curves, and capability indices are only as good as the history behind them. This guide shows how to backfill legacy results through the runs API with their original test dates intact. The key mechanism: a run's `started_at` is the timestamp analytics use, and the API accepts past dates. Upload date is stored separately, so a run tested in 2024 and imported today lands in 2024 on every chart. ## Prerequisites - An API key (Settings, API keys) - The procedure created in TofuPilot, and its procedure id - Legacy results exported to something parseable (CSV, database dump, report files) ## Step 1: Map the legacy fields The minimum viable mapping: | Legacy field | Runs API field | |---|---| | Test date/time | `started_at`, `ended_at` (ISO 8601) | | Overall verdict | `outcome` (PASS, FAIL, ERROR) | | Unit serial number | `serial_number` | | Part number | `part_number` | | Test steps | `phases[]` with name, outcome, timestamps | | Measured values and limits | `measurements[]` per phase, with validators | Keep phase and measurement names identical to what the live bench will record going forward. Analytics group by name, so `Contact Drop A2` in the import and `contact_drop_a2` from the new bench would land in two different series. ## Step 2: Create runs with original timestamps ```python filename="import_legacy.py" from tofupilot import TofuPilot client = TofuPilot(api_key="...") for row in legacy_rows: client.runs.create( procedure_id="550e8400-e29b-41d4-a716-446655440000", serial_number=row.serial, part_number=row.part_number, outcome=row.verdict, # "PASS" / "FAIL" started_at=row.tested_at, # original date, ISO 8601 ended_at=row.finished_at, phases=[{ "name": "Contact Voltage Drop", "outcome": row.phase_verdict, "started_at": row.tested_at, "ended_at": row.finished_at, "measurements": [{ "name": "Contact Drop A2", "outcome": row.a2_verdict, "measured_value": row.a2_mv, "units": "mV", "validators": [{"operator": "<=", "expected_value": 100, "outcome": row.a2_verdict}], }], }], ) ``` Units are created automatically from serial numbers, so a serial with five historical visits ends up with a five-run history without any separate unit setup. ## Step 3: Verify in analytics Set the date range in run analytics or measurement control to cover the imported period and check three things: 1. Run counts per month match the legacy system 2. Per-serial history is complete: pick a unit with known repeat visits and confirm every visit is there 3. Limits came through: measurements show their validators, so Cpk and pass rates compute against the right limits ## Step 4: Cut over the live bench Once history is verified, point the live bench at the same procedure. New runs continue the same series: the first live run on an imported serial extends its existing trend rather than starting a new one, which is the entire point of importing with real timestamps. Two details worth knowing: - Imported runs record the API key's user as creator, and the upload date as `created_at`. Both are visible on the run detail page; neither affects analytics, which use `started_at`. - Station attribution requires uploading with a station key. For historical data this rarely matters, but record the original site or bench as a measurement or run metadata if you need to filter by it later. ### Detect Latent Defects with Fleet Test Data URL: https://www.tofupilot.com/guides/detect-latent-defects-with-fleet-test-data Learn how to find units that pass every individual limit but carry a latent defect, using a derived measurement and fleet-level control charts. A latent defect is a unit that passes every individual limit on the bench but fails in service. This guide shows how one derived measurement plus a fleet-level control chart makes that population visible, using a simulated avionics contactor fleet as the worked example. The pattern matters most in maintenance and repair environments: high mix, low volume, complex degradation. A per-unit OK/NG verdict answers "can this unit ship today", but it says nothing about which passing units are quietly degrading. That signal only exists at fleet level. ## Prerequisites - A test procedure uploading runs to TofuPilot - A measurement that captures the defect signature (see Step 1) - Two or more shop visits per unit, so per-serial trends exist ## Step 1: Record a signature measurement Individual limits are set per measurement, so a defect that shifts the *relationship* between measurements stays invisible to them. Record that relationship as its own measurement. In the contactor example, each pole's voltage drop has a 100 mV limit. A latent coil defect raises one pole about 45 mV above the others while all three stay under 100 mV. The imbalance between poles is the signature: ```python filename="phases/contact_voltage_drop.py" # Imbalance between poles is a better early indicator than any single pole. spread = round(max(drops.values()) - min(drops.values()), 2) measurements.contact_drop_spread = spread ``` Give the derived measurement its own limit (40 mV here). One line in the bench script, and every past and future run carries the signature. ## Step 2: Read the fleet distribution Open measurement control, select the derived measurement over the full history, no other filters. A healthy fleet produces one population. A fleet with a latent defect mode produces two. In the contactor fleet (150 units, 24 months, 297 measurements), the histogram is bimodal: the healthy body sits near 11 mV, and a second population sits at 42-55 mV, above the 40 mV limit. Cpk on the same view (0.65 here) quantifies what the histogram shows. ![Fleet control chart of contact drop spread showing a bimodal distribution: healthy units near 11 mV and a defect population above the 40 mV limit](https://bfusqmdwknceg6ih.public.blob.vercel-storage.com/latent-defects-fleet-spread.jpg-370aab47-3f41-48ce-ad6b-782efdbf5db7-eE3PxVhhRF43WoLy58I9cJqVWWQpLT.jpg) No hypothesis was needed: selecting the measurement is the entire analysis. This is the practical difference from spreadsheet work, where each candidate relationship must be extracted, pivoted, and plotted one at a time. ## Step 3: Separate latent from overt units Filter the measurement outcome to FAIL and compare with the per-pole measurements: | Population | Definition | Contactor fleet | |---|---|---| | Overt | At least one individual limit failing | 14 units | | Latent | Every individual limit passing, signature over limit | 9 units | The 9 latent units are the ones a per-pole bench ships. They are also the future no-fault-found cases: removed on a squawk, tested OK on every individual limit, returned to stock. ## Step 4: Scope one serial and read the trend Filter to a single serial with the serial number filter and select the drifting measurement. On the example unit, pole A2 climbs linearly from 92.5 mV to 104.6 mV across six shop visits as operating hours accumulate, crossing its 100 mV limit between visits four and five. The spread measurement had been over its limit since the first visit, 19 months earlier. ![Control chart scoped to one serial: contact drop A2 climbing linearly across six shop visits and crossing its 100 mV limit](https://bfusqmdwknceg6ih.public.blob.vercel-storage.com/latent-defects-ctr-000001-a2.jpg-9e4f078b-dd93-4467-b3f6-15ff69de90bc-AaLW3lxjyXaZxv4VHhgaMNjhsYZbbL.jpg) On paper this unit is an intermittent repeat visitor. On the chart it is monotonic degradation with a predictable crossing date, which changes the maintenance decision from "test again and return to stock" to "replace the coil now". ## Step 5: Watch for it automatically Two alert types cover this pattern without manual review: - A **retest threshold** alert fires when a unit accumulates repeated tests on the same procedure, which is how no-fault-found loops look in the data. - A **measurement drift** alert grades each measurement against its own baseline and fires when the series genuinely moves, catching the per-serial climb of Step 4 as it happens. ## Templates ### IMU Thermal Calibration URL: https://www.tofupilot.com/templates/imu-thermal-calibration Improve your IMU's accuracy by calibrating for temperature changes. Includes automated thermal cycling, drift compensation, and production-ready test scripts. ## Introduction ### IMU Overview IMUs (Inertial Measurement Units) are remarkable sensors that measure the **movement** and **orientation** of the devices they are embedded in. Since the rise of smartphones, their size and cost have been drastically reduced, enabling a wide range of applications in robotics and drones, where they enable **autonomous navigation and guidance**. IMUs combine a **gyroscope** (measuring angular velocity in °/s or rad/s) and an **accelerometer** (measuring linear acceleration in m/s² or g). Gyroscopes measure rotation but can drift over time, while accelerometers sense velocity changes and tilt but cannot distinguish between motion and gravity. **Sensor** **fusion algorithms** correct gyroscope drift with accelerometer data and stabilize accelerometer noise with gyroscope input, ensuring accurate motion tracking. At the heart of an IMU is a **Micro-Electro-Mechanical System** (MEMS): a tiny structure, often just a few microns in size (about 1/100th the width of a human hair), that moves slightly in response to forces. These movements generate changes in voltage, which are measured by the sensor, converted into numerical data, and transmitted to your system through communication protocols such as SPI or I²C. ![Close-up of a MEMS-based IMU component showing micro-scale mechanical elements for motion sensing.](https://bfusqmdwknceg6ih.public.blob.vercel-storage.com/templates/da36041c763820b2563512c3064cdfdc22523633-1032x720-1wJmp0qpECQF7wqr9XhfH8pj1PzHmI.png) Micro-scale mechanical elements within a MEMS-based Inertial Measurement Unit (IMU). For everyday devices like smartphones or tablets, precision isn't as important, so IMUs are not necessary factory-calibrated for temperature. But for drones and robots used in outdoor conditions, accuracy matters and **calibrating the IMU for temperature** can greatly improve its performance. ### Calibration Purpose Thermal calibration involves placing the IMU-equipped **Printed Circuit Board Assembly** (PCBA) in a **climate chamber** for several hours while varying the temperature. With the IMU kept flat and motionless, any changes in its measurements are attributed to temperature effects. The resulting data is used to calculate and **save calibration parameters** specific to each board. During operation, these parameters are applied in real time to compensate for temperature-induced measurement variations. ![Graph displaying a polynomial fit curve used for thermal calibration of an IMU, showing the relationship between temperature and compensation values.](https://bfusqmdwknceg6ih.public.blob.vercel-storage.com/templates/f32725a889a9ad84c6ec1bcad3f8e2103caa1665-900x506-nz2tcm0BSX1Hth5DDzkpzJgQMVTxcl.png) Curve showing accelerometer X vs. temperature with a polynomial fit for calibration. Most **Electronic Manufacturing Service** (EMS) providers have climate chambers for tasks like stress testing, so reserving one for your boards shouldn't be an issue. For lab use, you can purchase a small climate chamber for under $5,000, suitable for testing a few boards. The cost largely depends on the chamber's **temperature range**. For instance, testing a drone designed for outdoor use may require a chamber that operates between **-20°C and 70°C** to simulate extreme environmental conditions. Note that the temperature measured by the IMU will always be higher than the chamber's temperature due to the internal heat generated by the IMU's casing and the PCBA it is mounted on. ## Equipment & Setup To implement thermal calibration for the IMU in a drone, you will need the following: * **A climate chamber** capable of reaching temperatures between **-20°C and 70°C**. * **A support structure** to hold the PCBA in the chamber and a power supply. * **A Device Under Test (DUT)**, equipped with the IMU that requires calibration. * **Firmware for the device**, with a triggerable mode to log the raw IMU data. * **A TofuPilot Framework procedure** to: * Retrieve data from each board after the calibration process. * Calculate the calibration parameters. * Verify the quality of the calibration and ensure there are no defects. * Save the calibration parameters to the product. * The **TofuPilot Dashboard** to store calibration data for traceability and analytics. ### Hardware Components #### Climate Chamber We will use the **Votsch VT4002** temperature test chamber, which has a temperature range of **-40°C to +130°C**. With its **16-liter volume**, it is ideal for designing the test in the lab or for small production runs. ![Votsch VT4002 test chamber with a rectangular design, viewing window, side control panel, and stainless steel exterior.](https://bfusqmdwknceg6ih.public.blob.vercel-storage.com/templates/4602a575d031f2ebfe7cf9d4570402908c205623-2400x1800-rTbvnyUOuSb5JtMXOp12ZlOvPJDSWW.png) The Votsch VT4002 is a compact lab test chamber with a -40°C to +130°C range. #### Cycle Program The IMU's response is not necessarily the same when it heats up versus when it cools down. To ensure the calibration accurately reflects the sensor's real thermal behavior, we will set a **calibration duration of 2 hours**, with **4 temperature cycles** ranging from **-20°C to 70°C**. #### Support Structure For the lab setup, we'll create a simple **3D-printed support** using **ESD filament**. Foam will be added to reduce vibrations from the climate chamber. While these vibrations are likely filtered out during data processing, minimising them is still beneficial. In mass production, we can collaborate with the **test team at our EMS** to design a support that accommodates multiple boards to optimise throughput, provides power to them, and is adapted to the dimensions of their temperature test chambers. ### Custom Firmware Throughout the entire calibration process, the IMU needs to log its measurements at a frequency of at least 10 Hz. To enable this, we will develop a **special logging mode** in the firmware. When activated, this mode will start data acquisition and record the following parameters: * **Timestamp**: To track when each measurement was taken. * **Gyroscope data**: X, Y, and Z axes in degrees per second (deg/s). * **Accelerometer data**: X, Y, and Z axes in meters per second squared (m/s²). * **Internal sensor temperature**: This will be used as the reference for calibration. The logged data can be stored as a **JSON or CSV file** in the PCBA memory or on an SD card. Most climate chambers have an **external pin** that activates at program start. Connecting this pin to a stabilized power supply outside the chamber allows the supply to **switch on automatically** at the program's start and **off at its end**, ensuring logging occurs only during the thermal cycle and not afterward, such as when the operator retrieves the board. ## Test Procedure ### Overview Once the thermal cycle is complete, the chamber powers off and the log file is ready for retrieval and processing. Operators will remove the PCBAs from their calibration support and connect them **to the test station**. At this point, the TofuPilot procedure takes over to: 1. Connect to the Device Under Test (DUT) and retrieve the acquisition file. 2. Validate the acquired data (noise density, temperature sensitivity). 3. Compute the polynomial calibration. 4. Validate the calibration quality (residuals, R²). 5. Save calibration results to the DUT internal memory. 6. Provide a global pass/fail status. 7. Stream results to TofuPilot for traceability and analytics. ### Why TofuPilot Framework? TofuPilot Framework is a **YAML + Python** test framework built for hardware manufacturing. Instead of writing all your test logic, measurements, and limits inside Python code, you describe **what** the test does in a `procedure.yaml` file, and **how** in small Python phase files. The framework handles: * Automatic Python environment management (via `uv`) * Operator UI (no frontend code needed) * Measurement validation and live charts * Process isolation between phases and equipment plugs ### Project Structure The whole procedure is six small files plus the data sample: ```tree procedure.yaml phases/ connect_dut.py thermal_calibration.py plugs/ mock_dut.py utils/ calibrate_sensor.py compute_noise_density.py compute_r2.py compute_residuals.py compute_temp_sensitivity.py data/ imu_raw_data.csv pyproject.toml ``` You can find the full source on [GitHub](https://github.com/tofupilot/template-framework-imu-thermal-calibration). ### The Procedure File `procedure.yaml` is the entry point. It declares: * The unit being tested (auto-identified here, so the run starts without operator input) * A **plug** (`Mock DUT`) that simulates the board * Two **phases**: `Connect DUT` and `Thermal Calibration` * All the **measurements** with their limits Here's the top of the file: ```yaml filename="procedure.yaml" name: IMU Thermal Calibration version: 0.1.0 description: Computes per-axis polynomial thermal compensation for an IMU's accelerometer and gyroscope, then validates fit quality. unit: auto_identify: true serial_number: default_value: "SN00001" part_number: default_value: "PCB01" plugs: - name: Mock DUT description: Simulated device under test that returns logged IMU CSV data. python: plugs.mock_dut:MockDut key: dut main: - name: Connect DUT key: connect_dut python: phases.connect_dut - name: Thermal Calibration key: thermal_calibration python: phases.thermal_calibration depends_on: - connect_dut ``` `python: plugs.mock_dut:MockDut` tells TofuPilot to instantiate the `MockDut` class. `python: phases.connect_dut` tells it to call the `connect_dut()` function in that file. The order of phases is enforced by `depends_on`. ### Mock DUT Plug A plug is a persistent Python class for a device or service. The framework creates it once at the start of the run and tears it down at the end. For this template we use a mock that reads a CSV file instead of talking to real hardware: ```python filename="plugs/mock_dut.py" import time from pathlib import Path import pandas as pd CSV_PATH = Path(__file__).resolve().parent.parent / "data" / "imu_raw_data.csv" class MockDut: """Simulated DUT that returns IMU log data from a CSV file.""" def __init__(self): self._connected = False print("Mock DUT initialized") def connect(self) -> bool: print("Connecting to mock DUT...") time.sleep(0.2) self._connected = True return True def get_imu_data(self) -> dict: df = pd.read_csv(CSV_PATH, delimiter="\t") return { "acc_data": { "temperature": df["imu.temperature"].tolist(), "acc_x": df["imu.acc.x"].tolist(), "acc_y": df["imu.acc.y"].tolist(), "acc_z": (df["imu.acc.z"] - 9.80600).tolist(), }, "gyro_data": { "temperature": df["imu.temperature"].tolist(), "gyro_x": df["imu.gyro.x"].tolist(), "gyro_y": df["imu.gyro.y"].tolist(), "gyro_z": df["imu.gyro.z"].tolist(), }, } def save_accelerometer_calibration(self, coefficients: dict) -> None: print(f"Saved accelerometer calibration: {list(coefficients.keys())}") def save_gyroscope_calibration(self, coefficients: dict) -> None: print(f"Saved gyroscope calibration: {list(coefficients.keys())}") ``` To run against a real board, swap this class for one that talks to your firmware (UART, USB CDC, network). The rest of the pipeline stays the same. ### Connect Phase The first phase is a one-liner. The framework injects the `dut` plug and a `log` object by matching parameter names: ```python filename="phases/connect_dut.py" def connect_dut(dut, log): log.info("Connecting to DUT...") dut.connect() log.info("DUT connected") ``` ### Calibration Phase The second phase does the real work. It retrieves the IMU log, validates it, fits per-axis polynomials, validates the fit quality, and saves the result. The framework automatically passes `dut`, `measurements`, and `log` based on the function signature: ```python filename="phases/thermal_calibration.py" import numpy as np from utils.calibrate_sensor import calibrate_sensor from utils.compute_noise_density import compute_noise_density from utils.compute_r2 import compute_r2 from utils.compute_residuals import compute_residuals from utils.compute_temp_sensitivity import compute_temp_sensitivity def thermal_calibration(dut, measurements, log): """Retrieve IMU data, validate it, compute polynomial thermal calibration, validate fit, save.""" log.info("Fetching IMU log from DUT") data = dut.get_imu_data() axes = ("x", "y", "z") calibration_results = {} for sensor, data_key in (("acc", "acc_data"), ("gyro", "gyro_data")): sensor_data = data[data_key] temperature = np.asarray(sensor_data["temperature"], dtype=float) axes_data = {axis: np.asarray(sensor_data[f"{sensor}_{axis}"], dtype=float) for axis in axes} # --- Raw-data validation --- for axis, values in axes_data.items(): noise = compute_noise_density(values) sens = compute_temp_sensitivity(values, temperature) setattr(measurements, f"{sensor}_noise_density_{axis}", noise) setattr(measurements, f"{sensor}_temp_sensitivity_ref_{axis}", sens["sensitivity_at_ref"]) # --- Polynomial calibration --- fit = calibrate_sensor((temperature, *axes_data.values())) calibration_results[sensor] = { axis: fit["polynomial_coefficients"][f"{axis}_axis"].tolist() for axis in axes } # --- Multi-dimensional chart per axis: raw / fitted / residual vs temperature --- order = np.argsort(temperature) temp_sorted = temperature[order] for axis in axes: raw = axes_data[axis][order] fitted = fit["fitted_values"][f"{axis}_axis"][order] residuals_dict = compute_residuals(raw, fitted) md = getattr(measurements, f"{sensor}_calibration_{axis}") md.x_axis = temp_sorted.tolist() md.y_axis.raw = raw.tolist() md.y_axis.fitted = fitted.tolist() md.y_axis.residual = residuals_dict["residuals"].tolist() aggs = md.y_axis.residual.aggregations aggs.mean = residuals_dict["mean_residual"] aggs.std = residuals_dict["std_residual"] aggs.p2p = residuals_dict["p2p_residual"] setattr(measurements, f"{sensor}_r2_{axis}", compute_r2(raw, fitted)) log.info("Saving calibration to DUT") dut.save_accelerometer_calibration(calibration_results["acc"]) dut.save_gyroscope_calibration(calibration_results["gyro"]) ``` The pattern is the same for every measurement: compute a value, write it to `measurements`. The framework matches it back to the YAML declaration and validates it. ### Numeric Measurements Each scalar measurement is declared in `procedure.yaml` with a name, unit, and validators. Validators express the pass/fail limits: ```yaml filename="procedure.yaml" measurements: - name: Acc Noise Density X key: acc_noise_density_x unit: m/s²/√Hz validators: - {operator: ">=", expected_value: 0.0} - {operator: "<=", expected_value: 0.003} ``` In Python you simply assign: ```python setattr(measurements, "acc_noise_density_x", noise) ``` The dashboard automatically renders each measurement with its limits, value, and pass/fail outcome. As more units are tested, **3 sigma limits** can be computed automatically from production data on the procedure analytics page. ### Multi-Dimensional Measurements For each sensor axis, we capture the **raw data**, the **fitted curve**, and the **residuals** as a single multi-dimensional measurement. This replaces the static PNG plots you would otherwise have to attach manually: ```yaml filename="procedure.yaml" - name: Acc Calibration X key: acc_calibration_x title: Accelerometer X vs Temperature description: Raw accelerometer X readings vs internal temperature, with fitted 3rd-order polynomial and residuals. x_axis: {legend: Temperature, unit: "°C"} y_axis: - {legend: Raw, key: raw, unit: "m/s²"} - {legend: Fitted, key: fitted, unit: "m/s²"} - legend: Residual key: residual unit: "m/s²" aggregations: - type: mean validators: - {operator: ">=", expected_value: -0.01} - {operator: "<=", expected_value: 0.01} - type: std validators: - {operator: "<=", expected_value: 5.0} - type: p2p validators: - {operator: "<=", expected_value: 15.0} ``` Three things to notice: 1. **`x_axis` / `y_axis`** describe the chart. The dashboard renders an interactive plot — no need to generate or attach a PNG. 2. **`aggregations`** compute statistics over an axis (mean, standard deviation, peak-to-peak) and validate them. Here the residual mean must be near zero, the standard deviation small, and the peak-to-peak bounded. 3. **R²** (declared separately as a numeric measurement) catches a globally poor fit. In Python, you set the data with intuitive attribute access: ```python md = measurements.acc_calibration_x md.x_axis = temperatures md.y_axis.raw = raw_values md.y_axis.fitted = fitted_values md.y_axis.residual = residuals aggs = md.y_axis.residual.aggregations aggs.mean = residual_mean aggs.std = residual_std aggs.p2p = residual_p2p ``` ### Polynomial Calibration The calibration logic itself lives in `utils/calibrate_sensor.py` and uses a **3rd-order polynomial** fit per axis. The coefficients replace the need for large lookup tables and are programmed into the device for real-time correction: ```python filename="utils/calibrate_sensor.py" import numpy as np def calibrate_sensor(data, polynomial_order: int = 3): """Fit a polynomial model per axis. Returns coefficients and fitted values.""" temp, *sensor_data = (np.asarray(arr, dtype=float) for arr in data) poly_coeffs = {} fitted_values = {} axis_list = ("x", "y", "z") for i, axis_data in enumerate(sensor_data): axis_name = f"{axis_list[i]}_axis" coeffs = np.polyfit(temp, axis_data, polynomial_order) poly_coeffs[axis_name] = coeffs fitted_values[axis_name] = np.polyval(coeffs, temp) return { "polynomial_coefficients": poly_coeffs, "fitted_values": fitted_values, } ``` ### Calibration Validation #### Residuals After fitting, we compute the **residuals** (the difference between the model's prediction and the actual measurements) and validate three statistics: * **mean** to detect systematic bias * **std** to detect variability across the temperature range * **peak-to-peak** to bound the worst-case error ```python filename="utils/compute_residuals.py" import numpy as np def compute_residuals(data, fit_model): residuals = np.asarray(data) - np.asarray(fit_model) return { "residuals": residuals, "mean_residual": float(np.mean(residuals)), "std_residual": float(np.std(residuals)), "p2p_residual": float(np.ptp(residuals)), } ``` #### Coefficient of Determination (R²) Residual metrics focus on local accuracy. **R²** evaluates how well the model represents the sensor's behavior globally — close to 1 means the model explains most of the variance, near 0 means poor fit: ```python filename="utils/compute_r2.py" import numpy as np def compute_r2(data, fit_model): data = np.asarray(data, dtype=float) fit_model = np.asarray(fit_model, dtype=float) residuals = data - fit_model total_variation = float(np.sum((data - np.mean(data)) ** 2)) if total_variation == 0.0: return 1.0 return 1.0 - float(np.sum(residuals ** 2)) / total_variation ``` ## Database & Analytics Sensor calibration, while complex, can be implemented quickly using TofuPilot Framework. The procedure can be developed during the validation phase of a new product and deployed in production when mass manufacturing begins. The quality of a test relies not only on the performance of the processing algorithms but also heavily on the **choice of metrics** used to validate the measurements. These metrics improve as more units are tested, allowing for more precise 3 sigma limits to be defined. This is where TofuPilot's database and analytics solution become essential. ### Automatic Upload When a procedure declares a Dashboard ID in `procedure.yaml`, every run is uploaded automatically — including phases, measurements, multi-dimensional charts, and logs. No extra code, no output callback, no station server. ### Run Page After a run completes, a dedicated page is **automatically created** in your secure TofuPilot workspace. This page displays the **test metadata** (serial number, run date, procedure reference), the list of **phases** and **measurements** with their **limits**, **units**, **duration** and **status**, plus all the multi-dimensional charts rendered interactively. ![Screenshot of the Run page showing a test report with detailed phases, key measurements, and metrics.](https://bfusqmdwknceg6ih.public.blob.vercel-storage.com/templates/cb2524d62672b52653d8cb3bea41cb405961b2c7-1920x1110-sxDweTMiKJdQUjhb6ik4NnHBVS3XVv.png) Run page displaying detailed test reports with phases and measurements. ### Procedure Analytics Analyzing the performance of a procedure across recent runs is straightforward with the **procedure analytics page**. Key metrics such as the **run count**, **average test time**, **first pass yield**, and **CPK** are calculated automatically. You can **filter the data by date, revision, or batch** to narrow down your analysis. The page also provides a **detailed breakdown of performance by phase and measurement**, allowing you to select a specific phase or measurement to analyze its **duration**, **first pass yield**, or **CPK** individually. Additionally, you can view its **control chart** to track recent measurements, observe trends, and determine **3 sigma values**. ![Screenshot of Procedure page showing metrics, filters, phase details, and control charts.](https://bfusqmdwknceg6ih.public.blob.vercel-storage.com/templates/89e23aa010ad30410d4ea58bafdf84e0b4e103f2-1920x1110-t4JzhqmRR0VVb09KWjxseNVP8dJtNl.png) Procedure page with performance metrics, filters, and control charts. ### Unit Traceability Finally, the traceability of each tested unit is easily accessible through its dedicated page. This page provides the complete **history of tests** performed for the unit, any related **sub-units**, and a link to the page dedicated to the **revision of the part**. ![Screenshot of the Unit page showing the test history and details for a specific unit.](https://bfusqmdwknceg6ih.public.blob.vercel-storage.com/templates/7a58aa49a348a7e51f7743bacaf3c6d4175fa91f-960x555-jmEWsRgK61thB3hnHs0buakOhNxi5f.png) Unit page displaying the complete test history of a tested unit. ## Changelog ### Improve chart readability and date filtering URL: https://www.tofupilot.com/changelog/improve-chart-readability-and-date-filtering Charts lacked axis labels and there was no easy way to filter data by date range. - Added x-axis labels to control chart and histogram - Added calendar and date range picker for filtering data - Moved search params to page header for cleaner layout ### Improve dashboard stability URL: https://www.tofupilot.com/changelog/improve-dashboard-stability Several UI issues were surfaced during customer demos. - Fixed multiple front-end errors affecting dashboard interactions - Improved overall page stability ### Track measurements at the phase level URL: https://www.tofupilot.com/changelog/track-measurements-at-the-phase-level Until now, measurements were only available at the run level. Phase-level tracking gives you finer-grained visibility into each step of a test. - Added phase-level measurements replacing the legacy report view - Added automatic extraction of part number and revision from serial numbers ### Manage test stations from the dashboard URL: https://www.tofupilot.com/changelog/manage-test-stations-from-the-dashboard Teams needed a central place to create, organize, and monitor their test stations. - Added full station management: create, update, delete, and archive - Added station activity tracker showing current runs per station ### Explore the API with built-in documentation URL: https://www.tofupilot.com/changelog/explore-the-api-with-built-in-documentation Integrating with TofuPilot is easier when you can browse the API directly from the app. - Added interactive API documentation with auto-generated schema - Added API call tracking so you can monitor usage and debug integrations ### Get help without leaving the app URL: https://www.tofupilot.com/changelog/get-help-without-leaving-the-app Reaching support required navigating to an external page. Now you can get help directly from the dashboard. - Added in-app support dialog with streamlined help request form - Added quick links to API documentation and resources ### Analyze multi-dimensional measurements URL: https://www.tofupilot.com/changelog/analyze-multi-dimensional-measurements Some tests produce measurements across multiple dimensions. You can now capture and visualize all of them. - Added support for multi-dimensional measurements - Added temporal graphs for phases and improved Cpk charts - Improved control chart display and measurement filtering ### Upgrade to Pro with integrated billing URL: https://www.tofupilot.com/changelog/upgrade-to-pro-with-integrated-billing Teams can now unlock advanced features by upgrading directly from the dashboard. - Added Pro subscription with integrated billing - Added subscription management with upgrade and downgrade flows ### Monitor tests in real time with the operator UI URL: https://www.tofupilot.com/changelog/monitor-tests-in-real-time-with-the-operator-ui Operators on the factory floor need live visibility into running tests without switching between tools. - Added real-time operator UI with live data streaming - Added operator UI support in Python scripts - Improved connection reliability and API navigation ### Navigate the dashboard with a redesigned layout URL: https://www.tofupilot.com/changelog/navigate-the-dashboard-with-a-redesigned-layout The app layout has been redesigned for faster navigation and a cleaner experience. - Redesigned app layout with improved navigation structure - Redesigned procedure cards and subscription badge - Reorganized settings pages with cleaner URLs ### Sign in with Microsoft, Google, and GitHub URL: https://www.tofupilot.com/changelog/sign-in-with-microsoft-google-and-github Users signing in with enterprise identity providers experienced login issues. All sign-in methods now work reliably. - Fixed Microsoft Entra ID and Google OAuth login flows - Added account linking for multiple auth providers ### Deploy TofuPilot on your own infrastructure URL: https://www.tofupilot.com/changelog/deploy-tofupilot-on-your-own-infrastructure Enterprise teams with strict data residency requirements can now run TofuPilot on-premise. - Added self-hosting deployment with Docker - Added configuration for on-premise instances ### Visualize measurement trends over time URL: https://www.tofupilot.com/changelog/visualize-measurement-trends-over-time Understanding how measurements evolve across test runs helps catch regressions early. - Added temporal graph visualization for phase measurements over time - Improved Cpk chart calculations and multi-unit comparison views ### See live test activity per station URL: https://www.tofupilot.com/changelog/see-live-test-activity-per-station Teams wanted to know which tests are running on each station right now. - Added station activity page showing current and past test runs - Added real-time current runs display per station ### Load insights page faster URL: https://www.tofupilot.com/changelog/load-insights-page-faster The insights page was slow to load for teams with large datasets. - Improved page load speed on the insights page - Fixed cycle time chart and batch filter issues ### Browse runs with infinite scroll and logs URL: https://www.tofupilot.com/changelog/browse-runs-with-infinite-scroll-and-logs Navigating large lists of test runs was slow, and there was no way to explore logs. This update makes both much easier. - Added infinite scroll pagination on the Runs page - Added log explorer with filtering, sorting, and color-coded levels - Added log support for Python vanilla scripts ### Add phases to existing runs via the Python SDK URL: https://www.tofupilot.com/changelog/add-phases-to-existing-runs-via-the-python-sdk You can now append phase data to runs that were already uploaded, making incremental test reporting easier. - Added API endpoint to add phases to existing runs - Added automatic testing of Python scripts before releases ### Retrieve run attachments by serial number URL: https://www.tofupilot.com/changelog/retrieve-run-attachments-by-serial-number When querying runs by serial number, attachment data was missing from the response. - Added attachment data to the get runs by serial number endpoint - Fixed response format to match documentation ### Launch the documentation site URL: https://www.tofupilot.com/changelog/launch-the-documentation-site All TofuPilot documentation is now in one place with full-text search and auto-generated API reference. - Added documentation site with markdown support and full-text search - Added auto-generated API documentation ### Integrate faster with API v2 URL: https://www.tofupilot.com/changelog/integrate-faster-with-api-v2 The new API version simplifies integrations with cleaner endpoint paths and a consistent response format. - Added API v2 with new endpoint paths and response format - Simplified runs list endpoint for cleaner integrations ### Load Cpk charts faster on large datasets URL: https://www.tofupilot.com/changelog/load-cpk-charts-faster-on-large-datasets Cpk charts were slow to render for organizations with thousands of measurements. - Improved Cpk chart computation performance - Removed redundant calculations when all values are identical - Added sorted values in Cpk chart filter ### Keep using get_runs with API v1 compatibility URL: https://www.tofupilot.com/changelog/keep-using-get-runs-with-api-v1-compatibility Teams using the v1 API needed continued access to the get_runs method during migration. - Restored legacy get_runs method in Python client for v1 API compatibility ### Improve date display and image handling URL: https://www.tofupilot.com/changelog/improve-date-display-and-image-handling Date formatting was inconsistent across the app, and some images failed to load in API responses. - Unified date formatting and display across the app - Fixed date range picker behavior for all range options - Improved image handling in API responses ### Fix revision settings page URL: https://www.tofupilot.com/changelog/fix-revision-settings-page Opening revision settings returned errors for some users. - Fixed 404 error when opening revision settings - Fixed permission errors when accessing revisions via API ### Get more detail from control charts URL: https://www.tofupilot.com/changelog/get-more-detail-from-control-charts Control charts now show richer information on hover, making it easier to investigate individual data points. - Added detailed tooltips showing unit and run data on control charts - Improved error states with contextual messaging ### Discover TofuPilot with new landing page URL: https://www.tofupilot.com/changelog/discover-tofupilot-with-new-landing-page The landing page and website have been refreshed to better showcase what TofuPilot does. - Redesigned landing page with dark mode support - Added SEO metadata across docs and web apps - Updated about page and navigation ### List units programmatically with get_units URL: https://www.tofupilot.com/changelog/list-units-programmatically-with-get-units You can now retrieve your unit inventory directly from your Python scripts. - Added get_units endpoint to the Python SDK ### Fix GitLab repository picker URL: https://www.tofupilot.com/changelog/fix-gitlab-repository-picker Fixed an issue where linking a GitLab repository could fail with a generic "Failed to fetch metadata" error. ### Guide operators with checklists URL: https://www.tofupilot.com/changelog/guide-operators-with-checklists Station procedures can now include visual checklists and require operator input at specific steps. - Added image choice and image checklist components for station procedures - Added requires_input toggle for operator prompts during test execution ### Distribute self-hosted via Docker URL: https://www.tofupilot.com/changelog/distribute-self-hosted-via-docker Self-hosted customers now receive Docker images gated by their license, simplifying deployment and updates. - Added license-gated Docker image access for self-hosted customers - Added automated Docker image builds for dev and production releases ### Fix part filtering on Batches and Units pages URL: https://www.tofupilot.com/changelog/fix-part-filtering-on-batches-and-units-pages Filtering by part on the Batches and Units pages caused errors for some organizations. - Fixed errors when filtering by part on Batches and Units pages - Fixed permission-related errors affecting filtered views ### Improve login reliability URL: https://www.tofupilot.com/changelog/improve-login-reliability Some users experienced authentication issues preventing them from signing in. - Fixed authentication errors affecting specific user accounts - Improved user tracking for newly signed up users ### Show accurate API error messages URL: https://www.tofupilot.com/changelog/show-accurate-api-error-messages API error responses sometimes showed unrelated messages, making debugging harder than it needed to be. - Fixed API error messages to show the actual error - Improved error clarity across all endpoints ### Speed up run creation by 17% URL: https://www.tofupilot.com/changelog/speed-up-run-creation-by-17 We migrated to a faster database engine, resulting in faster test uploads and better reliability under load. - Improved run creation speed by 17% - Completed migration with zero data loss and zero downtime ### Strengthen security across all services URL: https://www.tofupilot.com/changelog/strengthen-security-across-all-services We performed a comprehensive security review and patched all identified issues. - Updated all dependencies to latest secure versions - Hardened security configuration across all services ### Share filtered views via URL URL: https://www.tofupilot.com/changelog/share-filtered-views-via-url You can now bookmark or share a filtered page — the filters are preserved in the URL. - Pages now load with correct filtered data from URL search params - Fixed filters to always refresh data when changed ### Control test execution with phase dependencies URL: https://www.tofupilot.com/changelog/control-test-execution-with-phase-dependencies Complex test procedures need phases to run in a specific order with retries and timeouts. Studio now supports all of that. - Added phase dependencies, setup/teardown nesting, timeouts, and retries - Added operator UI with real-time component rendering - Improved execution bar with scope selectors and kill hotkey ### Run Studio reliably on Windows and Linux URL: https://www.tofupilot.com/changelog/run-studio-reliably-on-windows-and-linux Several platform-specific issues were preventing smooth test execution on Windows and Linux. - Fixed console windows appearing during test execution on Windows - Fixed Python environment initialization on Linux - Fixed deep links on Windows desktop builds ### Get started faster with automatic Python setup URL: https://www.tofupilot.com/changelog/get-started-faster-with-automatic-python-setup Setting up a Python environment was a manual and error-prone step. Studio now handles it automatically. - Switched to automatic Python environment management - Added auto-created project files for new projects - Added signed Linux packages for secure distribution ### Run Studio on ARM64 Linux devices URL: https://www.tofupilot.com/changelog/run-studio-on-arm64-linux-devices Teams deploying on edge devices like NVIDIA Jetson Nano can now run Studio natively. - Added ARM64 Linux build target - Expanded platform coverage for edge deployment scenarios ### Choose which version to deploy URL: https://www.tofupilot.com/changelog/choose-which-version-to-deploy Self-hosted customers previously always got the latest version. You can now pin a specific version during deployment. - Added version selector to the deploy script - Users can now choose specific versions instead of always installing the latest ### Upgrade from v1 to v2 safely URL: https://www.tofupilot.com/changelog/upgrade-from-v1-to-v2-safely Upgrading between major versions can be risky. The new upgrade script handles the migration with built-in safety checks. - Added upgrade script migrating from v1 to v2 database and storage - Added safety features to prevent data loss during migration - Added partial upgrade recovery for failed attempts ### Run OpenHTF tests with real-time operator UI URL: https://www.tofupilot.com/changelog/run-openhtf-tests-with-real-time-operator-ui OpenHTF users can now execute tests through Studio with live operator feedback and automatic dashboard uploads. - Added OpenHTF test runner integration with real-time operator UI - Connected Studio execution engine with dashboard upload pipeline ### Browse Framework and API docs in one place URL: https://www.tofupilot.com/changelog/browse-framework-and-api-docs-in-one-place All documentation — Framework guides, API reference, and examples — now lives on a single site. - Added docs site with landing page and support resources - Added Framework documentation with structure, phases, and plugs - Improved operator UI and plug documentation with examples ### Learn Framework faster with better docs URL: https://www.tofupilot.com/changelog/learn-framework-faster-with-better-docs The Framework docs have been restructured to help you get productive faster. - Restructured docs with environments and execution sections - Added download cards for all platforms - Updated template covers and configuration ### Sync procedures from Git, localize Studio URL: https://www.tofupilot.com/changelog/sync-procedures-from-git-localize-studio Teams storing procedures in Git can now sync them directly into Studio. The entire UI is also now translatable. - Added GitHub/GitLab repository sync for cloning repos locally - Completed i18n migration of all UI strings and error messages - Renamed "runs" to "reports" with improved report UI ### Launch Studio with editor and execution URL: https://www.tofupilot.com/changelog/launch-studio-with-editor-and-execution Studio is a new desktop app for creating and running test procedures. Define your phases in YAML, execute them, and stream results in real time. - Added procedure editor with phase configuration - Added test execution with real-time log streaming - Available on macOS with signed and notarized builds ### Aggregate data and validate test results URL: https://www.tofupilot.com/changelog/aggregate-data-and-validate-test-results New chart views help you spot trends across runs, and validators let you define pass/fail criteria for measurements. - Added aggregation and validator views to the dashboard - Fixed phase timestamp display for legacy data - Added string validator compatibility for migrated stations ### Fix charts crashing on large datasets URL: https://www.tofupilot.com/changelog/fix-charts-crashing-on-large-datasets Organizations with many parts experienced chart crashes when loading histograms or filtering. - Fixed histogram and sidebar filters crashing on large datasets - Added smart limits to chart series and lazy rendering for filter panels ### Connect GitLab repositories for test uploads URL: https://www.tofupilot.com/changelog/connect-gitlab-repositories-for-test-uploads Teams using GitLab can now connect their repositories to TofuPilot, just like GitHub users. - Added GitLab as a supported Git provider alongside GitHub - Enabled seamless token refresh for GitLab connections ### Deploy and upgrade self-hosted instances faster URL: https://www.tofupilot.com/changelog/deploy-and-upgrade-self-hosted-instances-faster Repeated deployments are now faster and more reliable, with better error handling throughout. - Sped up repeated deploys by skipping already-completed steps - Fixed authentication secret generation and environment variable handling - Added clear error messages and fail-fast on migration errors ### Manage self-hosted versions with v0.1.1 URL: https://www.tofupilot.com/changelog/manage-self-hosted-versions-with-v011 The deploy script now shows your current version and detects upgrades or downgrades automatically. - Added version display with upgrade/downgrade detection in deploy script - Added automatic cleanup of old Docker images after successful deploy - Fixed version listing with authenticated registry access ### Run tests locally or upload to the cloud URL: https://www.tofupilot.com/changelog/run-tests-locally-or-upload-to-the-cloud Studio now supports two modes: desktop mode for local-only testing (no API key needed) and cloud mode for uploading results to your dashboard. - Added desktop mode and cloud mode for test execution - Improved error reporting when test scripts fail ### Use pytest and Robot Framework in Studio URL: https://www.tofupilot.com/changelog/use-pytest-and-robot-framework-in-studio Studio now supports pytest and Robot Framework alongside OpenHTF, so you can use your preferred test framework. - Added pytest and Robot Framework as test framework connectors - Enabled real-time test status, phase tracking, and log streaming ### Download Studio for Apple Silicon and ARM Linux URL: https://www.tofupilot.com/changelog/download-studio-for-apple-silicon-and-arm-linux Studio is now available as native ARM64 builds for Apple Silicon Macs and ARM-based Linux devices. - Added ARM64 builds for Apple Silicon and ARM-based Linux devices - Updated download page with new platform options ### Sort unit lists 12x faster URL: https://www.tofupilot.com/changelog/sort-unit-lists-12x-faster Sorting large unit lists was slow for teams with thousands of units, causing noticeable delays when navigating. - Reduced unit list sort time from 518ms to 43ms - Biggest improvement for organizations with 15,000+ units ### Set up self-hosted for air-gapped envs URL: https://www.tofupilot.com/changelog/set-up-self-hosted-for-air-gapped-envs Self-hosted installations now work in restricted environments without internet access or SMTP configuration. - Added Azure Government cloud authentication support - Made analytics and email sending optional for air-gapped environments - Added "Share invite link" dialog when SMTP is not configured ### Control access with teams and roles URL: https://www.tofupilot.com/changelog/control-access-with-teams-and-roles Growing teams need fine-grained control over who can do what. You can now assign roles and restrict access per team. - Added team-based access control with admin, member, and viewer roles - Added support for email, Google, GitHub, and Microsoft Entra ID sign-in - Added station API key authentication for automated test uploads ### Improve mobile layout and team creation URL: https://www.tofupilot.com/changelog/improve-mobile-layout-and-team-creation The procedures list is now responsive on mobile, and team creation respects your subscription tier. - Fixed team creation button to respect subscription tier - Made procedures list responsive for mobile - Improved page load speed across the app ### Upgrade to Python SDK v2.0 URL: https://www.tofupilot.com/changelog/upgrade-to-python-sdk-v20 The Python SDK v2.0 aligns with the new API v2, bringing a simpler authentication flow and updated endpoint paths. - Migrated to new API key authentication scheme - Updated all endpoint paths for v2 API compatibility ### Launch tofupilot.com URL: https://www.tofupilot.com/changelog/launch-tofupilotcom The new tofupilot.com website is live with a product overview and staff-only access to internal tools. - Added landing page with product overview and hero section - Added authentication with staff-only access for internal tools ### Navigate the website with a cleaner structure URL: https://www.tofupilot.com/changelog/navigate-the-website-with-a-cleaner-structure The website has been reorganized into clearer sections with direct navigation links. - Restructured pages into route groups for products, resources, and company - Updated navbar with direct links instead of dropdown menus - Aligned hero section width with content layout ### Use TofuPilot in your language URL: https://www.tofupilot.com/changelog/use-tofupilot-in-your-language The website is now available in multiple languages, and the Orbit product page has a fresh new look. - Added internationalization for landing, pricing, about, orbit, and footer pages - Redesigned Orbit hero with new illustration and layout ### Refresh email branding URL: https://www.tofupilot.com/changelog/refresh-email-branding Emails from TofuPilot now feature the updated mascot icon and branding. - Replaced email header logo with new TofuPilot mascot icon - Updated email template to use com domain for assets ### Improve website SEO and page performance URL: https://www.tofupilot.com/changelog/improve-website-seo-and-page-performance Search engines and users now find TofuPilot pages faster with optimized metadata and improved load times. - Optimized meta descriptions across 77 pages and consolidated sitemap - Fixed canonical URLs, heading structure, and broken links for better crawlability - Added dynamic social preview images and improved page load speed ### Access docs at tofupilot.com/docs URL: https://www.tofupilot.com/changelog/access-docs-at-tofupilotcomdocs All documentation now lives under the main tofupilot.com domain for a unified experience. - Unified documentation under the main tofupilot.com domain - Fixed version mismatches across documentation pages ### Fix docs page loading errors URL: https://www.tofupilot.com/changelog/fix-docs-page-loading-errors Some documentation pages were intermittently failing to load. - Fixed intermittent loading errors on documentation pages - Improved docs routing reliability ### Simplify self-hosting setup documentation URL: https://www.tofupilot.com/changelog/simplify-self-hosting-setup-documentation Getting started with self-hosting is now a single command instead of cloning a repository. - Replaced git clone instructions with one-line curl deploy command - Added GitHub and GitLab OAuth provider setup guides - Added version management and uninstall flags documentation ### View and track your quotes in the customer hub URL: https://www.tofupilot.com/changelog/view-and-track-your-quotes-in-the-customer-hub Customers can now view and track their quotes directly from the Orbit portal. - Added customer hub for viewing and tracking quotes - Redesigned Orbit hero illustration ### Show the real cause of CLI connection errors URL: https://www.tofupilot.com/changelog/show-the-real-cause-of-cli-connection-errors When a station could not reach its TofuPilot instance, the CLI printed a generic "error sending request" message with no way to tell what actually failed. - Fixed connection errors to report the underlying cause, such as a rejected TLS certificate, a DNS failure or a proxy issue - Added the full error chain to the message, for example "invalid peer certificate: UnknownIssuer" when the instance certificate is not trusted ### Fix license error messages during deployment URL: https://www.tofupilot.com/changelog/fix-license-error-messages-during-deployment The self-hosted deployment script showed a generic "Invalid license key" error for all failures, making it difficult to diagnose connection problems versus actual license issues. - Fixed the deployment script to show specific error messages: malformed key format, unrecognized key, expired or suspended license, or network connectivity problems - Improved error messages for expired or suspended licenses to display the reason from the licensing server ### Ship test data from Rust URL: https://www.tofupilot.com/changelog/ship-test-data-from-rust You can now push test runs to TofuPilot from Rust. The SDK covers the full V2 API with async builders, typed errors, retries, and file upload helpers. ```rust let client = TofuPilot::new("your-api-key"); client.runs().create() .procedure_id("FVT-001") .serial_number("SN-042") .part_number("PART-001") .outcome(Outcome::Pass) .send() .await?; ``` Source on [GitHub](https://github.com/tofupilot/rust), package on [crates.io](https://crates.io/crates/tofupilot). ### Track phase retries in test runs URL: https://www.tofupilot.com/changelog/track-phase-retries-in-test-runs When a test phase fails and is retried, you can now store every attempt with a `retry_count` field and filter retries out of your analytics. - Added `retry_count` to phases: set it to track which attempt each phase represents (0 = first, 1 = first retry). Defaults to 0 when omitted. - Added "Exclude retries" toggle on all insights charts (Cpk, cycle time, failures, control charts, duration) to focus on final attempts only - Retried phases show expandable attempt tabs in the phase detail view - OpenHTF retries are detected automatically from repeated phase names, no changes needed in test scripts ### Improve API reliability and performance URL: https://www.tofupilot.com/changelog/improve-api-reliability-and-performance Rebuilt our REST API layer for better error handling, faster request routing, and more accurate OpenAPI documentation. This improves integration stability for all API and SDK users. ### Deploy test procedures from GitLab URL: https://www.tofupilot.com/changelog/deploy-test-procedures-from-gitlab Connect your GitLab repositories to automatically deploy test procedures to your stations and trace every test run back to its source code. - Added GitLab integration supporting both GitLab.com and self-hosted instances - Added automatic procedure deployment to stations when pushing to GitLab - Added webhook-based synchronization to keep branches and commits in sync ### Upload test results from MATLAB URL: https://www.tofupilot.com/changelog/upload-test-results-from-matlab You can now upload runs, manage parts, and query test data directly from MATLAB -- no middleware, no CSV exports. - Upload runs with measurements, limits, and attachments in a few lines of MATLAB code - Query and filter units, batches, and procedures from your test scripts - Automatic retries on transient errors so uploads don't fail mid-production - Ships as a .mltbx toolbox -- install once, available across all your projects ### Process control for every measurement URL: https://www.tofupilot.com/changelog/process-control-for-every-measurement The Insights page is now Process Control, rebuilt from scratch. - Added interactive control charts with spec limits, control limits, histograms, and capability indices (Cp, Cpk, Pp, Ppk) for numeric, string, and boolean measurements - Added sidebar filters for outcome, value range, retries, and all standard run filters, with a measurement selector sorted by failures, fail rate, or Cpk - Added measurement table with checkbox selection to drill down into matching runs or units in one click - Faster loads. Charts and capability indices now compute server-side. ### Attach files to runs and units in one call URL: https://www.tofupilot.com/changelog/attach-files-to-runs-and-units-in-one-call Uploading files to test runs and units used to require three API calls (initialize, PUT, finalize) plus a fourth to link the attachment. Now it's one call across all five SDK clients. - Added `runs.attachments.upload()` and `units.attachments.upload()` helpers that handle the full upload flow in a single call across Python, C#, Rust, C++, and MATLAB - Added `units.attachments.delete()` to remove attachments from units by ID - Added `runs.attachments.download()` and `units.attachments.download()` to download attachments to local files ```python from tofupilot.v2 import TofuPilot client = TofuPilot() # Upload a file to a run client.runs.attachments.upload(id=run.id, file="data/test-report.pdf") # Upload a file to a unit client.units.attachments.upload(serial_number="SN-0001", file="data/calibration.pdf") # Download client.runs.attachments.download(attachment, dest="local-report.pdf") # Delete (units only) client.units.attachments.delete(serial_number="SN-0001", ids=[attachment_id]) ``` ### Stations v2.0 URL: https://www.tofupilot.com/changelog/stations-v20 A headless-first rewrite of TofuPilot Stations: more hardware architectures supported, real-time control from the web, isolated artifacts, and push-to-deploy with environments. - Rebuilt on a headless core that runs on Raspberry Pi (arm64), Linux (amd64/arm64), Windows, and macOS: pick a kiosk UI, terminal UI for SSH and edge boxes, or no UI at all - Streamed every station live to the web for remote control and telemetry, with the offline queue and full Framework UI components carried over from v1 - Replaced repo-pull with a managed artifact system so stations never touch your Git provider directly - Auto-deployed on merge to your configured environments, with skippable preview environments matched by branch pattern - Run TofuPilot Framework procedures end-to-end from your repo to your stations ### Better monorepo venv support URL: https://www.tofupilot.com/changelog/better-monorepo-venv-support Stations now find the right Python environment every time, including monorepo deployments where the venv lives next to a sub-package. - Improved monorepo support: venv is resolved deterministically at the package directory, no more walk-up search - Fixed run failures on stations where divergent venv resolvers picked the wrong interpreter - Added regression tests covering monorepo, single-package, and local-path runs ### Add V2 Python SDK examples to docs URL: https://www.tofupilot.com/changelog/add-v2-python-sdk-examples-to-docs All dashboard documentation pages now include working Python V2 SDK code examples. V1 client syntax has been removed and replaced with `tofupilot.v2` throughout. ### Add strongly-typed validators to all SDKs URL: https://www.tofupilot.com/changelog/add-strongly-typed-validators-to-all-sdks All SDK clients now expose typed validator objects instead of untyped JSON, making it easier to build and validate measurement validators with IDE autocomplete and compile-time checks. - Simplified validators API schema for cleaner OpenAPI output, removing legacy string union type - Regenerated C++, Rust, C#, and Python SDK clients with strongly-typed validator structs and builders - Fixed C# generator to support CancellationToken in all async methods and synced SDK version to 2.3.0 - Fixed Speakeasy generation script to use correct config and synced version to 2.2.2 ### Fix workflow trigger accuracy URL: https://www.tofupilot.com/changelog/fix-workflow-trigger-accuracy Workflows could incorrectly trigger from unrelated events. - Fixed workflow execution to correctly fire only on matching events - Improved organization-level filtering in workflow endpoints ### Fix SPC sidebar filters with multiple values URL: https://www.tofupilot.com/changelog/fix-spc-sidebar-filters-with-multiple-values Fixed a bug that could cause SPC sidebar filters to silently ignore selections when more than one value was picked. Multi-value filters now narrow the chart as expected. ### Override the entry point per procedure URL: https://www.tofupilot.com/changelog/override-the-entry-point-per-procedure Procedures can now declare which file or directory the station hands to the framework runner. Useful when your repo doesn't follow the default layout, or when you want pytest to scan a specific subdirectory. - Added an Entry point field to the Build configuration section on every procedure page - Forwarded the value through the deployment manifest so the station picks it up on the next pull - Defaulted to the framework convention when left blank: main.py for OpenHTF and plain Python, the package directory for pytest, procedure.yaml for the TofuPilot Framework ### Detect pytest repos without a main.py URL: https://www.tofupilot.com/changelog/detect-pytest-repos-without-a-mainpy Pytest-only repos with no sentinel main.py now show up in the New Procedure picker on their own. The audit treats a leaf test_*.py next to pyproject.toml as enough signal to commit to the pytest framework. - Added test_*.py / *_test.py as a procedure signal in the repo audit - Defaulted detected pytest procedures to entry point "." (the package directory) - Dropped the sentinel main.py from the Pytest Starter template ### Default new stations to kiosk mode URL: https://www.tofupilot.com/changelog/default-new-stations-to-kiosk-mode Operator-facing stations come up in kiosk mode out of the box, with a tighter install command and a polished repo list on the New Procedure flow. - Defaulted new station deployments to kiosk mode (full-screen operator UI on boot) - Refactored the install command snippet so the copy-paste flow is shorter - Polished the linked-repo list on the procedure detail page ### Self-hosting license management URL: https://www.tofupilot.com/changelog/self-hosting-license-management TofuPilot now handles license validation, plan enforcement, and usage tracking for self-hosted instances directly in the dashboard. - Added feature-based license gating. SSO, SCIM, and Teams are now tied to your plan tier and enforced at runtime. - Built a self-hosted usage page showing station count, user count, and monthly runs against plan limits. - Added air-gapped license activation for instances without internet access. - License card on the plan page shows current plan, limits, and expiry at a glance. - Fixed station duplicate check and limit-reached modal that blocked test uploads silently. ### Export and copy data from list pages URL: https://www.tofupilot.com/changelog/export-and-copy-data-from-list-pages You can now export data from any list page without writing a query. - Added Copy, Export to CSV, and Export to JSON on Runs, Units, Logs, and API Activity tables. - Added the same options in the selection bar to copy or download just the rows you pick. ### Fix datetime filters and simplify finalize URL: https://www.tofupilot.com/changelog/fix-datetime-filters-and-simplify-finalize You can now pass `datetime` objects to date filters on `units.list()` and `batches.list()`. Previously these only accepted strings, while `runs.list()` already handled `datetime`. All list endpoints are now consistent. We also cleaned up `attachments.finalize()`. It no longer asks for an empty `request_body` parameter. If you were passing `request_body={}`, you can drop it. ### Auto-append /api to custom server URLs URL: https://www.tofupilot.com/changelog/auto-append-api-to-custom-server-urls All SDK clients now automatically append `/api` to custom server URLs, so self-hosted servers no longer need to include it manually — passing `https://demo.tofupilot.sh` now works out of the box without getting redirected to the signin page. ### Compare measurements on SPC charts URL: https://www.tofupilot.com/changelog/compare-measurements-on-spc-charts Control charts were limited to one measurement at a time. You can now overlay multiple compatible measurements on a single chart to spot correlations and trends across your process. - Added multi-measurement selection with Shift-click on the measurement overview, with compatibility checks (same type and units) - Added phase filter in sidebar showing data point counts per phase when multiple measurements are selected - Added compatibility tooltips showing which criteria match or mismatch when selecting measurements - Limits and Cpk tab now hide automatically when selected measurements have different spec limits ### Redesign run sheet measurement table URL: https://www.tofupilot.com/changelog/redesign-run-sheet-measurement-table The phase/measurement table on the run sheet page got a full redesign. Measurements now open in a dedicated sidebar panel with charts, validators, and aggregations. - Restyled phase/measurement table with consistent row spacing, sticky header, and unified column grid matching other tables across the app - Added measurement and phase sidebar panels with full detail views: metadata, validators table, aggregations table, JSON code block, and multi-dimensional charts with reference lines - Migrated multi-dimensional charts to recharts with legend, axis labels, toggleable aggregation reference lines, and a fullscreen dialog - Added outcome filters (fail, error, skip), search, sort, and relative/duration time toggle to the section header ### Add named data series and multi-axis charts URL: https://www.tofupilot.com/changelog/add-named-data-series-and-multi-axis-charts Multi-dimensional measurements now support named data series with independent units, enabling richer time-series visualizations. - Added `name` field to data series, allowing each axis to carry a label (e.g. "Frequency", "Voltage") - Moved `units` from measurement level to individual data series for per-axis unit control - Added multi-axis Y chart support with independent scales when series have different units - Added interactive series toggle in chart legend to show/hide individual data series ### Deploy from uv-workspace monorepos URL: https://www.tofupilot.com/changelog/deploy-from-uv-workspace-monorepos Procedures living inside a uv-workspace monorepo can now be deployed without restructuring the repo, by pointing the CLI at the package directory. - Added a `package_directory` setting on procedures so deployments resolve dependencies from a sub-package instead of the repo root. - Updated the CLI and deployer to walk up from the package directory to find the workspace venv and lockfile. - Consolidated the deploy pipeline behind a typed v1 manifest, with framework auto-detected from disk at runtime. ### Fix measurement outcome rollup URL: https://www.tofupilot.com/changelog/fix-measurement-outcome-rollup Measurements with a recorded value but no validators were reaching the dashboard as `UNSET` instead of `PASS`, mostly affecting string measurements used for traceability. - Fixed outcome rollup to match OpenHTF semantics: null value stays `UNSET`, any failing validator forces `FAIL`, otherwise `PASS`. - Added unit tests covering the parity rules. ### Reliability and kiosk fixes URL: https://www.tofupilot.com/changelog/reliability-and-kiosk-fixes A round of fixes covering the auto-update flow, kiosk polish on Windows, the git provider setup, and how stations refresh to the latest CLI. - Fixed auto-update segfaults and the boot ENOENT race that occasionally crashed stations on first launch after an upgrade - Fixed the Windows TUI double-registering keystrokes and the desktop shortcut launching the procedure runner instead of the station daemon - Fixed the GitHub and GitLab provider setup screens losing state on reload, and surfaced underlying provider errors instead of a generic toast - Refreshed installs to the latest CLI faster: removed the CDN cache on the version endpoint and suppressed spurious "update failed" warnings on subsequent boots ### Fix billing seat count to exclude banned users URL: https://www.tofupilot.com/changelog/fix-billing-seat-count-to-exclude-banned-users Organizations were being overcharged because banned users were incorrectly counted as active seats in billing calculations. - Fixed billing to exclude banned users from seat counts - Fixed unbanning users to correctly restore their seat in billing ### Sign Windows CLI builds with a code certificate URL: https://www.tofupilot.com/changelog/sign-windows-cli-builds-with-a-code-certificate Windows now trusts the TofuPilot CLI out of the box, removing the "unknown publisher" SmartScreen warning that appeared when running downloaded builds. - Signed the Windows CLI executable with a new code-signing certificate issued to TofuPilot SA - Verified every release build carries a valid signature before it ships ### Fix batch unit count for batches over 100 units URL: https://www.tofupilot.com/changelog/fix-batch-unit-count-for-batches-over-100-units Batch lists were showing incorrect unit counts for batches with more than 100 units, making it difficult to track production progress accurately. - Fixed batch unit counts to show the correct total instead of capping at 100 - Updated batch list API response to return `unit_count` as an integer - Removed unnecessary unit details from batch list responses to reduce payload size ### Fix missing API pages and add CLI snippets URL: https://www.tofupilot.com/changelog/fix-missing-api-pages-and-add-cli-snippets You can now copy `tofupilot` CLI commands straight from the API reference. - Added a CLI tab next to Python, cURL, and the other code samples on every endpoint. - Fixed missing endpoints and the empty cURL tab in the v2 reference. ### Clean up README and improve release automation URL: https://www.tofupilot.com/changelog/clean-up-readme-and-improve-release-automation The README contained outdated example code and placeholder text that didn't reflect the current v2 SDK, making it harder for new users to get started. - Updated README with working v2 SDK examples and removed boilerplate content - Added automatic release notes on the repository when new versions are published - Fixed version tag format to match existing convention ### Remove console banner and version check URL: https://www.tofupilot.com/changelog/remove-console-banner-and-version-check The Python SDK no longer prints a banner or checks PyPI on every client init — cleaner output, faster startup, no network call at import time. ### Speed up procedure analytics page URL: https://www.tofupilot.com/changelog/speed-up-procedure-analytics-page The procedure analytics page now loads faster, even on procedures with tens of thousands of runs. - Improved performance across charts, KPIs, and filter sidebars - Added a guardrail that prompts you to narrow the date range on very large queries ### GitLab real-time sync and soft delete URL: https://www.tofupilot.com/changelog/gitlab-real-time-sync-and-soft-delete GitLab sync relied on polling and deleted branches disappeared from deployment history, losing audit context. - Added real-time webhook sync for GitLab push and merge request events, replacing polling - Preserved deleted branches and commits in deployment history with visual indicators instead of removing them - Replaced OAuth setup with a token-first flow supporting both Group and Personal Access Tokens ### Add meetings section to Orbit URL: https://www.tofupilot.com/changelog/add-meetings-section-to-orbit Enterprise customers can now view their meetings with TofuPilot directly in Orbit, including agenda, summary, and linked action items with assignees. ### Ship test data from C# and .NET URL: https://www.tofupilot.com/changelog/ship-test-data-from-c-and-net You can now push test runs to TofuPilot from C# and .NET. The SDK covers the full V2 API with typed responses and errors. ```csharp var client = new TofuPilot(apiKey: "your-api-key"); await client.Runs.CreateAsync(new RunCreateRequest { ProcedureId = "FVT-001", SerialNumber = "SN-042", PartNumber = "PART-001", Outcome = RunCreateOutcome.Pass, }); ``` API docs now show C# examples next to Python. Source on [GitHub](https://github.com/tofupilot/csharp), package on [NuGet](https://www.nuget.org/packages/TofuPilot). Thanks to community member [@Hylaean](https://github.com/Hylaean) for building the original 1.x client that got this started. ### Add Rust code snippets to API reference URL: https://www.tofupilot.com/changelog/add-rust-code-snippets-to-api-reference Every API reference page now includes Rust code samples alongside Python, cURL, and C#. - Added Rust builder-pattern snippets for all 44 V2 API endpoints - Fixed C# snippet generation for add and remove sub-unit operations ### Rewrite self-hosting docs and deploy script URL: https://www.tofupilot.com/changelog/rewrite-self-hosting-docs-and-deploy-script The self-hosting documentation has been fully rewritten to match the actual deploy flow, and the deploy script now auto-generates a `.env` template on first run. - Rewrote self-hosting docs to cover the real deploy flow end-to-end - Deploy script auto-generates `.env` template when missing, instead of failing silently - Added documentation for license management, air-gapped activation, and Orbit monitoring - Merged Admin and Enterprise sidebar sections for clearer navigation - Fixed license refresh endpoint that caused 500 errors during validation - Fixed station duplicate check and limit-reached modal ### Improve process control and chart export URL: https://www.tofupilot.com/changelog/improve-process-control-and-chart-export Process control gets smarter defaults, better filtering, and a new PDF export for sharing charts outside TofuPilot. - Added A4 landscape PDF export for charts with title and "Exported by" attribution - Added validators column and measurement outcome filter to the process control measurement table - Added default 30-day date filter on runs, logs, and API calls pages for faster initial load - Fixed invisible connecting lines between points on control charts - Fixed missing procedure, phase, unit, and run outcome on measurement detail page ### Automate test actions with workflows URL: https://www.tofupilot.com/changelog/automate-test-actions-with-workflows Trigger automated actions when test events occur. Connect your test data to the tools your team already uses, without writing glue code. - Added visual workflow editor with drag-and-drop nodes, branching (if/else, switch), filters, and versioning - Added integrations for Odoo, Linear, InvenTree, Discord, Slack, email, and generic HTTP - Added execution history with per-step status, error details, and flow visualization ### Smoother GitHub setup on self-hosted URL: https://www.tofupilot.com/changelog/smoother-github-setup-on-self-hosted Self-hosted installations can now connect GitHub with clearer feedback at every step. - Repository picker lists all repositories from GitHub and GitLab - Connection errors surface clearly for easier troubleshooting ### Improve station connectivity diagnostics URL: https://www.tofupilot.com/changelog/improve-station-connectivity-diagnostics Faster troubleshooting when the kiosk page is unreachable or blank, with clearer log output pointing at the actual cause. - Added `tofupilot service status` loopback probe that reports listening, refused, or timeout, and names the process holding the port. - Added locale-stable Windows port-holder lookup so non-English Windows hosts (Chinese, Japanese, German, French) report the same diagnostics as macOS and Linux. - Fixed hardcoded port `7321` in several paths so `TOFUPILOT_LOCAL_UI_PORT` is honored everywhere. - Improved bind errors with specific remediation hints for port-in-use, permissions, loopback down, and container/network-namespace cases. - Added kiosk readiness probe and browser exit detection so blank-page situations surface in the logs instead of failing silently. ### Run Robot Framework suites on stations URL: https://www.tofupilot.com/changelog/run-robot-framework-suites-on-stations Robot Framework joins pytest and OpenHTF as a first-class test framework on TofuPilot stations. - Added a native Robot Framework connector: each test case becomes a phase, with measurements from the shipped TofuPilot keyword library - Detected .robot suites automatically when importing a repo, with a new Robot starter template - Added end-to-end coverage with eight Robot scenarios alongside the existing pytest agent-protocol tests ### Fix Dashboard Filters URL: https://www.tofupilot.com/changelog/fix-dashboard-filters Fixed an issue where some dashboard filters could return incorrect options after an internal infrastructure upgrade. Our compatibility audit caught the regression, and we've added preventive checks to our release process for future dependency upgrades. ### Cleaner runs across runtimes URL: https://www.tofupilot.com/changelog/cleaner-runs-across-runtimes Stations now produce consistent results whether running OpenHTF or pytest, and back-to-back runs no longer carry over leftover state. - Aligned operator UI behavior across both runtimes for parity in measurements and outcomes - Fixed cross-run state leaks so each run starts clean - Hardened protocol handling to prevent stale data between runs ### Upgrade server runtime to Node.js 24 URL: https://www.tofupilot.com/changelog/upgrade-server-runtime-to-nodejs-24 Upgraded our server runtime to the latest stable LTS version for better performance, security patches, and long-term support. - Upgraded server runtime from Node.js 20 to 22 across all deployments - Updated self-hosted Docker image to the new runtime ### Faster, more reliable Windows stations URL: https://www.tofupilot.com/changelog/faster-more-reliable-windows-stations Setting up and running stations on Windows is now smoother, with fewer prompts and quicker startup. - Fixed Windows station install so PowerShell execution policy no longer blocks setup - Pre-seeded kiosk procedures before launch so stations are ready immediately - Reduced teardown latency between runs on Windows ### Ship test data from C++ URL: https://www.tofupilot.com/changelog/ship-test-data-from-c Teams running C++ test scripts can now upload runs, measurements, and attachments to TofuPilot directly from their codebase. - Added full V2 API coverage for runs, units, parts, batches, stations, procedures, and attachments - Added builder-pattern API for creating runs with phases, measurements, validators, and logs - Added one-call file upload and download helpers for test reports and artifacts ### Fix part validation blocking unit creation URL: https://www.tofupilot.com/changelog/fix-part-validation-blocking-unit-creation Creating a unit from a part with `.`, `:`, or `+` in its number used to fail with a raw validation blob. Validation now matches what part creation allows, and errors are readable. - Fixed unit creation rejecting valid parts whose numbers contain `.`, `:`, or `+` - Improved error messages in modals to show a clear one-line reason instead of a raw JSON payload - Added a Copy action on error toasts so you can share the exact message with support ### Steadier station operation URL: https://www.tofupilot.com/changelog/steadier-station-operation Stations recover better from edge cases and surface clearer feedback when things go wrong. - Fixed dispatcher hangs so stations no longer get stuck between runs - Improved worker error reporting so failures are easier to diagnose - Cleaned up kiosk shutdown and update handling for orphaned stations ### Live updates without polling URL: https://www.tofupilot.com/changelog/live-updates-without-polling Runs, units, and logs pages now stream updates over a single realtime channel instead of polling every few seconds, so live mode is lighter and more reliable. - Replaced 3-second polling on list pages with realtime invalidation ### Mark units as golden or failing samples URL: https://www.tofupilot.com/changelog/mark-units-as-golden-or-failing-samples Reference units can now be tagged as known-good (golden) or known-faulty (failing) so they're excluded from production analytics and FPY/Cpk stay honest. - Added a sample classification dropdown on the unit detail page with activity tracking - Added a sidebar filter and table avatar treatment for golden and failing samples - Surfaced the sample field in V2 API for unit and run get/list/create/update ### Add custom metadata to runs and units URL: https://www.tofupilot.com/changelog/add-custom-metadata-to-runs-and-units Tag runs and units with your own typed key/value fields so traceability data, and qualification flags live next to the test data they describe. - Added a metadata editor on run and unit detail pages with type-aware icons and inline edit - Surfaced metadata in the V2 API as a plain `{key: value}` dict; types (string, number, bool) are detected automatically - Added metadata filters on run and unit list endpoints (`in`, `contains`, `gte`, `lte`, `eq`) - Tracked every metadata change in a per-entity activity timeline ### Add file attachments to units via Python SDK URL: https://www.tofupilot.com/changelog/add-file-attachments-to-units-via-python-sdk You can now attach files directly to units using the Python SDK v2.1.0. Upload photos, test reports, calibration data, or any file and link it to a unit in two lines of code. - Added `attachments.upload()` to handle file uploads in a single call - Added `attachments.download()` to download attachments from units or runs - Added attachments support to the Update Unit endpoint - Renamed the attachment finalization endpoint for clarity ### Rewrite test and device data model pages URL: https://www.tofupilot.com/changelog/rewrite-test-and-device-data-model-pages Both fundamentals data model pages were rewritten into one consistent reference with updated diagrams. - Rewrote the test and device data model pages with a uniform per-concept format. - Refreshed the animated diagrams and added a Schema reference section with direct links to REST API endpoints. ### Deploy self-hosted from GitHub workflows URL: https://www.tofupilot.com/changelog/deploy-self-hosted-from-github-workflows Self-hosted instances can now enable automatic deployments by installing the TofuPilot GitHub App. The setup process is documented in the [self-hosting guide](https://www.tofupilot.com/docs/dashboard/self-hosting). Cloud users already have this built in. ### Fix deployment commit filters URL: https://www.tofupilot.com/changelog/fix-deployment-commit-filters - Fixed commit list filters on the deployments page that weren't connected to commit data, causing confusing empty results. - Laid the server, worker, and UI foundations for the upcoming Station v2 release. ### Support timezone offsets in datetime fields URL: https://www.tofupilot.com/changelog/support-timezone-offsets-in-datetime-fields Timestamps with timezone offsets like `+02:00` or `+05:30` now work across all API endpoints. You no longer need to convert to UTC before uploading. - Accepted timezone offsets on all datetime input fields: run timestamps, phase and log timestamps, and date filters on runs, units, batches, and procedures - Responses still return UTC (`Z` suffix), no changes needed on the read side ```python from datetime import datetime, timezone, timedelta # UTC (always worked) client.runs.create( started_at=datetime.now(timezone.utc), ended_at=datetime.now(timezone.utc), ... ) # Local timezone (now works too) cet = timezone(timedelta(hours=2)) client.runs.create( started_at=datetime.now(cet), ended_at=datetime.now(cet), ... ) ``` ### Deploy OpenHTF procedures from GitHub URL: https://www.tofupilot.com/changelog/deploy-openhtf-procedures-from-github OpenHTF is a first-class framework alongside the TofuPilot Framework for stations: clone a starter from /new, push to GitHub, the station picks up your test on the next deploy. - Added a one-click OpenHTF Starter template in /new -- single procedure showing plugs, prompts, hard + marginal limits, regex validators, voltage-vs-time chart, attachments, and skip flow control - Streamed live phase events (phase started/ended, measurements, prompts, attachments) so the operator UI mirrors what is happening on the station in real time - Routed native user_input prompts to the web operator UI, with timeouts and operator-typed responses flowing back to OpenHTF without leaving the dashboard - Rendered marginal limits, multi-dim charts, and validator expressions natively in the dashboard from the connector's structured payloads ### Fix template clone auto-deploy URL: https://www.tofupilot.com/changelog/fix-template-clone-auto-deploy Fixed an issue when cloning a template: auto-push now picks up the seed commit so the procedure deploys to your stations immediately. ### Speed up the measurement control page URL: https://www.tofupilot.com/changelog/speed-up-the-measurement-control-page The measurement control page could freeze or take a long time to open for procedures with very large numbers of measurements. - Made control charts open instantly, even for measurements with millions of data points, while pass rate and Cpk stay accurate across every point - Made the value filter appear right away instead of waiting on heavy data - Added loading previews and smoother chart hover and selection ### C++ SDK gets typed metadata URL: https://www.tofupilot.com/changelog/c-sdk-gets-typed-metadata C++ client now ships the run + unit metadata feature with a typed `std::variant` for the value instead of a bare `nlohmann::json`. - Regenerated C++ SDK against the new metadata endpoints - Metadata field is now `std::map>` so values keep their type at compile time - Added Custom Metadata sections to the runs and units guides with snippets for Python, C#, C++, and MATLAB ### Add changelog filters and "What's new" in Orbit URL: https://www.tofupilot.com/changelog/add-changelog-filters-and-whats-new-in-orbit The changelog now supports URL-based filtering, and Orbit shows what changed between your current version and the latest. - Added changelog filtering by product and version range to quickly find relevant updates - Added "What's new" link in Orbit instance menus that shows releases between your current and latest available version ### Fix station deployment pulls after CLI deploy URL: https://www.tofupilot.com/changelog/fix-station-deployment-pulls-after-cli-deploy Stations that had also been used for `tofupilot deploy` could fail to pull deployments with a "Station authentication required" error, even right after logging in. Reinstalling didn't help. - Fixed `tofupilot pull` so it always uses the station identity, not a leftover user login from a deploy. - Added a clearer error message when a user login reaches a station-only command, pointing to the exact fix. ### Run pytest test suites on stations URL: https://www.tofupilot.com/changelog/run-pytest-test-suites-on-stations Pytest is now a first-class procedure framework alongside OpenHTF and the TofuPilot Framework. Each test function becomes a phase, and recognized assert shapes are promoted to measurements with validators on the dashboard, no SDK imports required. - Added a pytest connector that streams phase events live and bundles measurements per phase - Added AST extraction for numeric ranges, single bounds, equality, pytest.approx, string equality and membership, and boolean asserts - Added support for multiple measurements in one test (distinct identifiers) and stacked validators on one measurement (repeated asserts on the same identifier) - Added a Pytest Starter template in the New Procedure picker so you can clone and run in one click ### Improve database performance monitoring URL: https://www.tofupilot.com/changelog/improve-database-performance-monitoring TofuPilot now tracks database query performance, so slow pages and API responses get found and fixed faster. - Added query performance tracking to the TofuPilot cloud database - Updated self-hosted installations to enable the same tracking automatically at the next upgrade ### Add timezone-aware datetime hover cards URL: https://www.tofupilot.com/changelog/add-timezone-aware-datetime-hover-cards You can now hover any date to see UTC and local time side by side. - Added a rich hover card on date cells: live relative age, plus UTC and local timezone rows. - Replaced absolute/relative toggle on activity feeds with compact relative time; hover reveals full timestamps. ### Fix linking GitLab repos in nested subgroups URL: https://www.tofupilot.com/changelog/fix-linking-gitlab-repos-in-nested-subgroups Linking a GitLab project that lived under a subgroup failed with a generic "Failed to fetch metadata" error, even when the connected token had full access. The metadata lookup was double-encoding the project path before sending it to GitLab. - Fixed GitLab project lookups for any multi-segment path (e.g. `group/subgroup/project`) - Linking repositories deep in nested GitLab groups now succeeds on the first try - Branch listing, commit history, file content, and tree fetches across nested GitLab projects also resolve correctly ### Transitional fixes ahead of user-mode CLI URL: https://www.tofupilot.com/changelog/transitional-fixes-ahead-of-user-mode-cli Stop-gap fixes for the most common spawn failures we're seeing today, while we finish full user-mode support for the CLI in an upcoming release. - Fixed local runs on Windows with default antivirus settings - Surfaced clear error messages when the Python venv isn't usable - Reported a terminal event to the kiosk UI when a subprocess fails to start ### New Test Lifecycle page with animated diagram URL: https://www.tofupilot.com/changelog/new-test-lifecycle-page-with-animated-diagram A new fundamentals page maps the four stages of a hardware test on TofuPilot end to end. - Added a "Test Lifecycle" page covering author, deploy, run, and analyze with an animated diagram of the cycle - Listed the three authoring paths (TofuPilot Framework, supported framework, custom test code) at every stage so it is clear how each one moves through the lifecycle ### Documentation 2.0 URL: https://www.tofupilot.com/changelog/documentation-20 A re-architected documentation site with cleaner navigation and dedicated space for the new surface areas TofuPilot ships against. - Reorganized the sidebar around what you actually use day-to-day. Top groups now flow from "Getting started" and "Fundamentals" into product surfaces: Frameworks (TofuPilot, OpenHTF, Pytest, Robot), API & SDKs (Python, C#, Rust, C++, MATLAB, REST), CLI, AI, Analytics, Deployments, Integrations, Inventory, Station, Access, Security, Pricing, Self-hosting - Replaced the flat dump of pages with grouped sections, so framework-specific guidance, SDK references, and integration setups each live under their own dedicated group instead of sharing one bucket - Added per-resource sub-sections so growing surface areas (every CLI command, every REST endpoint, every Station toggle) have their own page instead of being concatenated - Added a Preview badge and banner so features still rolling out (AI, Slack, InvenTree, Reports, Insights, Failure Analysis, Rolling Rollout, Instant Rollback) are clearly flagged ### Add Run, Log, and Unit explorers URL: https://www.tofupilot.com/changelog/add-run-log-and-unit-explorers Three new explorer pages give the table-driven workflows you live in every day their own home in the docs. - Added Run, Log, and Unit explorer pages under Analytics, each covering columns, filters, saved views, export (CSV and JSON, visible or selected rows), metadata, and role-based permissions - Renamed the station Operator UI page to Remote UI to avoid confusion with the framework's per-phase operator UI components - Rewrote the docs landing around the actual journey: build, deploy, operate, observe, collaborate, integrate, then explore - Added a "Get support" section on the landing with direct links to Discord, support email, trainings, and custom integrations ### Animate test and device data model diagrams URL: https://www.tofupilot.com/changelog/animate-test-and-device-data-model-diagrams Two new animated tree diagrams make the test and device data models easy to read at a glance, with example values and pass/fail states streaming in as each entity is highlighted. - Added an animated tree on the Test Data Model page (Procedure, Run, Phase, Measurement, Validator, Log, Attachment) with live example values and a PASS validator outcome - Added an animated tree on the Device Data Model page (Part, Revision, Unit, Sub-unit, Batch) with the Batch drawn as a dashed connector to signal that batches are optional and cross-cutting - Renamed the underlying pages to Test Data Model and Device Data Model and updated every link across the docs ### Keep yield alerts open until quality recovers URL: https://www.tofupilot.com/changelog/keep-yield-alerts-open-until-quality-recovers A yield alert could close itself while the line was still failing. The drop that raised it slowly became part of the baseline it was being compared against, so a single quality incident produced a string of alerts opening and closing on their own — one incident on a production line generated nine. - Fixed yield alerts closing on the very failures that raised them, so one incident now stays one alert until someone resolves it - Locked each alert to the baseline measured when it was raised, so the bar to close it no longer drifts while the incident is still open - Applied the same rule to custom yield rules, which recalculated their reference window on every check ### Refresh doc links for docs v2.0 URL: https://www.tofupilot.com/changelog/refresh-doc-links-for-docs-v20 Following the docs v2.0 release, every doc link across the dashboard, marketing site, and docs site has been audited and pointed at the current pages so users land on the right place instead of a stale route. - Repointed dashboard empty states, deployment pages, and OpenAPI references to the new docs IA. - Updated landing footer, navbar, connector, and self-hosting links so the site sends visitors to the right docs. - Fixed internal docs redirects after the data-model rename and refreshed SDK readmes with the new per-language guide links. ### Document the run import API URL: https://www.tofupilot.com/changelog/document-the-run-import-api The REST API reference now covers run import, and every import format has a quick-reference spec at a glance. - Added reference pages for the import and batch import endpoints. - Added a spec block to each format page (OpenHTF, WATS, ATML, TestStand, STDF, ATDF, CSV, Excel) showing encoding, extension, detection, origin, and what it maps to. - Replaced the import endpoints table with cards matching the rest of the page. ### Rewrite incremental migration guide URL: https://www.tofupilot.com/changelog/rewrite-incremental-migration-guide The `/docs/incremental-migration` page is now structured around vertical and horizontal migration strategies, with an animated diagram that shows each one in action. - Reframed the page around incremental migration benefits, disadvantages of one-time migrations, and three strategies: vertical (one integration layer at a time across procedures), horizontal (one procedure end-to-end), and hybrid. - Added an animated grid that plays vertical fill first, pauses, then horizontal fill so the difference is visible at a glance. ### Fix bot author label on template deployments URL: https://www.tofupilot.com/changelog/fix-bot-author-label-on-template-deployments Fixed an issue where deployments from a new procedure template showed the TofuPilot bot as an "External Contributor". - Bot-authored template commits now show a "TofuPilot Bot" badge. ### Import runs from OpenHTF, WATS, and CSV files URL: https://www.tofupilot.com/changelog/import-runs-from-openhtf-wats-and-csv-files Bring your existing test reports into TofuPilot without writing any API code. Drag a file into the dashboard and it becomes a run, so you can backfill history from benches and testers you already have. - Added file import in the dashboard for OpenHTF JSON logs and WATS reports, auto-detected on upload, plus CSV files where you map your columns to fields once and save the mapping as a preset to reuse on every later file. Batch import handles many files at once. - Added duplicate detection, so re-importing the same file links to the existing run instead of creating a copy. - Improved the import and list pages with an inline drag-and-drop dropzone at the top of the import table, quick add buttons on the runs and units lists, and a clearer download icon for export. ### Filter runs and units by custom metadata URL: https://www.tofupilot.com/changelog/filter-runs-and-units-by-custom-metadata The runs and units pages now let you filter by any custom metadata you attach, and stay responsive even with tens of thousands of records. - Added per-key metadata filters that auto-discover the keys in use: pick string values from a list, set numeric min/max ranges, or toggle true/false. Combine several keys to narrow on all of them at once. - Applied the same metadata filter to the timeline chart so its counts match the table below. - Rebuilt list loading so a page is a fixed handful of queries regardless of row count: metadata filtering resolves in a single query instead of one per key, and revision images and avatars load in one batch instead of one request per row. - Fixed a bug where sorting units by part number, last run, or last procedure while a filter was active could fail to load results. ### Fix stuck deployment in new procedure wizard URL: https://www.tofupilot.com/changelog/fix-stuck-deployment-in-new-procedure-wizard The "Build first deployment" step in the clone-template and import wizards could spin forever even after the build finished successfully, leaving you unable to complete setup. - Fixed the wizard hanging on the build step by ensuring live build status always reaches the page - Added a fallback that detects a finished build within a second if a realtime update is missed - Prevented duplicate procedures and stations from a double-click or retrying after a failed build ### Clearer metadata filters on runs and units URL: https://www.tofupilot.com/changelog/clearer-metadata-filters-on-runs-and-units Custom metadata filters on the runs and units pages are now easier to read and stay in sync. - Added a divider separating standard filters from your custom metadata filters. - Metadata you add to a run or unit appears as a filter option immediately, no refresh needed. ### Split file import API by format URL: https://www.tofupilot.com/changelog/split-file-import-api-by-format The V2 import API now has a dedicated endpoint for each kind of file, so structured datalogs and tabular spreadsheets each get a request shape that fits them. - Added POST /v2/imports/structured to import 1 to 100 OpenHTF, WATS, ATML, NI TestStand, STDF, or ATDF files in one call, each parsed independently so one bad file never fails the rest. - Added POST /v2/imports/tabular to import a CSV or Excel file with an inline column mapping or a saved template. - Updated every SDK client (Python, C#, Rust, C++, MATLAB) with typed methods for both endpoints. ### Rewrite the file import guide URL: https://www.tofupilot.com/changelog/rewrite-the-file-import-guide The import documentation now explains every supported format, how to import from the dashboard or API, and how to convert an unsupported format yourself. - Added a clear breakdown of structured and tabular formats with a brand icon for each. - Added a section on writing a converter with the REST API, a typed SDK, or a coding agent when your format isn't supported natively. - Updated the API reference with the new structured and tabular import endpoints. ### Match procedure docs to actual CLI behavior URL: https://www.tofupilot.com/changelog/match-procedure-docs-to-actual-cli-behavior The procedure documentation now reflects what the CLI actually enforces, removing constraints that never existed. - Updated CLI and framework pages to say procedure files can use any `.yaml` name; `procedure.yaml` is only the default lookup when running a directory - Fixed the procedures page: names are not uniqueness-enforced and semantic versioning is a recommendation, not a requirement - Added the distinction between the version field you bump yourself and deployments built automatically from every commit ### Real-time and deployments for self-hosted URL: https://www.tofupilot.com/changelog/real-time-and-deployments-for-self-hosted Self-hosted instances now get the same live updates and build pipeline as the cloud, from a single prebuilt image on any domain. - Added a real-time WebSocket service so self-hosted dashboards show live station status, telemetry, and streaming build logs instead of manual refresh. - Added a build worker that compiles procedure bundles and stores them as deployment artifacts. - Improved setup with automatic DNS, TLS, secrets, and image checks; documented the new subdomain, requirements, and troubleshooting. ### Rename procedure link file to tofupilot.json URL: https://www.tofupilot.com/changelog/rename-procedure-link-file-to-tofupilotjson The file `tofupilot link` writes to bind a local procedure directory to the dashboard is now named after the product, so it is easier to recognize in your repo. - Renamed the link file from `procedure.json` to `tofupilot.json`; re-run `tofupilot link` in previously linked directories after updating - Improved error messages to reflect that procedure files can use any `.yaml` or `.yml` name, not only `procedure.yaml` - Clarified the no-procedure-found error to list everything the CLI looks for in a directory ### Speed up dashboard page loads URL: https://www.tofupilot.com/changelog/speed-up-dashboard-page-loads Navigating to the dashboard previously showed a blank page until account and organization data finished loading, and every page carried chart code it did not use. - Improved perceived load time: pages now paint instantly with loading skeletons while the sidebar streams in - Added loading skeletons to phase pareto, run analytics, workflows, and reports - Reduced the script payload shipped on every dashboard page by loading chart-based filters only on the routes that use them ### Fix real-time updates across page navigation URL: https://www.tofupilot.com/changelog/fix-real-time-updates-across-page-navigation Fixed an issue where real-time station online/offline status, dashboard updates, and build logs stopped connecting after navigating within the app. ### Fix self-hosted CLI install command in sidebar URL: https://www.tofupilot.com/changelog/fix-self-hosted-cli-install-command-in-sidebar Improved the station sidebar install command to include your instance URL on self-hosted deployments. ### Speed up measurement analysis to under 1s URL: https://www.tofupilot.com/changelog/speed-up-measurement-analysis-to-under-1s The measurement analysis view on high-volume procedures could take 38 seconds to load and sometimes timed out entirely. It now loads in under a second on the same data. - Added a daily measurement rollup that pre-aggregates pass/fail counts and Cpk, so the analysis view reads summarized data instead of scanning millions of raw measurements. - Improved load time on the heaviest procedures from 38 seconds (often a timeout) to under one second. - Kept results live and exact by combining the rollup with a real-time scan of the most recent runs. ### Fix deployment builds for GitLab repositories URL: https://www.tofupilot.com/changelog/fix-deployment-builds-for-gitlab-repositories Deployment builds failed for procedures linked to GitLab repositories, including self-hosted instances. - Fixed the build worker so it can clone GitLab repositories and produce deployments - Added a clear failure reason to build errors when a deployment cannot be cloned ### Fix value counts on boolean and string charts URL: https://www.tofupilot.com/changelog/fix-value-counts-on-boolean-and-string-charts On measurements with more than 500 records, the boolean and string control charts derived their value distribution from a sampled subset, so the True/False split and per-value counts could be slightly off. They now reflect the exact full dataset. - Fixed the True/False and string value counts to be computed over all records, not the chart's sampled points. - Kept the chart fast by computing the exact distribution server-side only when sampling is in effect. ### Control station auto-updates from dashboard URL: https://www.tofupilot.com/changelog/control-station-auto-updates-from-dashboard Stations previously installed CLI updates automatically with no way to opt out, which made update timing unpredictable on production lines. - Added the ability to turn automatic CLI updates on or off per station from the Setup panel - Fixed stations without a saved setting showing auto-update as off while updates were actually running ### Fix auto-deploy on first push from template URL: https://www.tofupilot.com/changelog/fix-auto-deploy-on-first-push-from-template Fixed an issue where auto-deploy did not trigger on self-hosted instances when GitHub omitted commit details from the push event, such as the first push after creating a repository from a template. - Fixed auto-deploy to fetch the pushed commit when the webhook payload omits it, so deployments trigger reliably. - Fixed self-hosted GitHub App commits not being recognized as trusted, so template-clone deployments run with the right permissions. ### Simplify organization member management URL: https://www.tofupilot.com/changelog/simplify-organization-member-management Managing who has access to your organization is now simpler and safer. - Replaced account suspension with a clear "Remove from organization" action that revokes access without touching the person's account elsewhere - Removed members keep their history: runs and records they created stay fully visible for traceability - Restoring access is now a simple re-invite ### Fix self-hosted builds on cgroup v2 hosts URL: https://www.tofupilot.com/changelog/fix-self-hosted-builds-on-cgroup-v2-hosts Fixed an issue where the self-hosted build worker could not start procedure builds on hosts using cgroup v2 (current Ubuntu and Debian releases). - Fixed the build sandbox so procedure builds run correctly on cgroup v2 hosts. ### Add instant rollback for deployments URL: https://www.tofupilot.com/changelog/add-instant-rollback-for-deployments Stations can be re-pinned to a previous deployment without a rebuild. - Added instant rollback: re-pins all stations to an older production build and pauses auto-push until you push again or resume it - Added a deployment row menu: push to all stations, redeploy, view source, copy URL - Added current and rolled back labels in the deployments list ### Lay groundwork for Failure Analysis URL: https://www.tofupilot.com/changelog/lay-groundwork-for-failure-analysis We're laying the groundwork for Failure Analysis, a new way to find which test phases fail most and dig into why. - Added the first building blocks ahead of a wider rollout. ### Guide v1.0 stations to upgrade to the CLI URL: https://www.tofupilot.com/changelog/guide-v10-stations-to-upgrade-to-the-cli Stations v2.0 run on the TofuPilot CLI. The dashboard and the desktop app now point v1.0 station users to the upgrade, with step-by-step guides. - Added a prompt on the desktop station app and the dashboard deploy page that explains the move from the v1.0 desktop station to the CLI. - Linked the migration guide everywhere, plus a self-hosted guide for enabling realtime and deploys when the prompt detects a self-hosted instance. - Marked the desktop station app as v1 so it is clear which stations still need to upgrade. ### Speed up page loads with session caching URL: https://www.tofupilot.com/changelog/speed-up-page-loads-with-session-caching Every page load and API call previously checked the session against the database before doing anything else. - Added short-lived signed session caching so pages and API calls skip one database round-trip on every request - Improved time to first paint on all dashboard pages ### Fix rollback offered when nothing is pushed URL: https://www.tofupilot.com/changelog/fix-rollback-offered-when-nothing-is-pushed Instant Rollback could run when no production deployment was pushed to any station, pausing auto-push on what was really a first push. - Fixed the rollback dialog to disable Rollback and point to Push to All Stations when no production deployment is currently pushed - Added a server check rejecting rollbacks with nothing to roll back from, so the procedure can no longer end up incorrectly pinned with auto-push paused ### Fix docs search returning no results URL: https://www.tofupilot.com/changelog/fix-docs-search-returning-no-results Search in the documentation was not returning results. - Fixed the docs search endpoint so queries return results again - Moved search server-side, replacing the full index download with small per-query responses ### Speed up measurement and log pages URL: https://www.tofupilot.com/changelog/speed-up-measurement-and-log-pages The measurement analysis and list views could take several seconds to load on high-volume procedures, and the logs page was slow on large log histories. A round of optimizations removes those slow queries so these pages stay fast as your data grows. - Reworked how the measurement views load data, cutting load times from seconds to milliseconds. - Sped up the logs page on large log histories. - Removed redundant queries by deriving sidebar filter counts from data the page already loads. - Changes apply automatically on both cloud and self-hosted instances, with no manual setup. ### Improve dashboard run links from the CLI URL: https://www.tofupilot.com/changelog/improve-dashboard-run-links-from-the-cli Run links shared by the CLI now point directly to the right run page. - Updated CLI dashboard run links to open the correct run page ### Fix sidebar back button losing procedure URL: https://www.tofupilot.com/changelog/fix-sidebar-back-button-losing-procedure Going back from a procedure-scoped page like measurement control no longer resets the selected procedure. - Fixed the sidebar back button to return to the procedure overview instead of the organization home when a procedure is selected - Applied the fix across all sidebar pages: runs, logs, units, batches, measurement control, and phase pareto ### Tidy up global search results URL: https://www.tofupilot.com/changelog/tidy-up-global-search-results Search results now carry only the links that are actually used, removing stale entries behind the scenes. - Removed unused run, unit, part, and batch links from search results ### Filter runs and units by metadata via the API URL: https://www.tofupilot.com/changelog/filter-runs-and-units-by-metadata-via-the-api Filter runs and units by their custom metadata, directly from the API and SDKs. - Added metadata filtering to the run and unit list endpoints with per-key operators (`in`, `contains`, `gte`, `lte`, `gt`, `lt`, `eq`) - Supported metadata filters across all SDKs and the CLI ### Add operator hints to Identify Unit fields URL: https://www.tofupilot.com/changelog/add-operator-hints-to-identify-unit-fields Guide operators with custom hints under each field on the Identify Unit screen, so they know exactly what to scan or enter. - Added an optional `description` to identify-unit fields (serial number, part number, revision, batch, sub-units) in the procedure config - Rendered the helper text under each field in both the CLI and the Station operator UI ### Patch dependency from routine security audit URL: https://www.tofupilot.com/changelog/patch-dependency-from-routine-security-audit Our routine dependency audit flagged a security issue, which we patched right away. - Updated an affected dependency to its patched release - No action needed on your end ### Launch Phase Pareto failure analysis URL: https://www.tofupilot.com/changelog/launch-phase-pareto-failure-analysis Find the weakest step in a procedure: Phase Pareto ranks phases by failures, retries, and duration so you target the real bottleneck instead of guessing. - Added a Phase Pareto chart with Failures, Retries, and Duration views, each with previous-period trend badges - Added phase-level filters for outcome, sample class (production / golden / failing), and full run and unit slicing, kept in sync between the sidebar and command bar - Added a retry-count breakdown that surfaces phases churning through repeated attempts - Improved the date filter to window on when each phase ran, not when its run started ### Fix station install command on test stations URL: https://www.tofupilot.com/changelog/fix-station-install-command-on-test-stations The station install command now uses an explicit secure URL, avoiding a redirect hop that could fail on older or proxied curl and leave test stations unable to install the CLI. - Fixed station install commands to use a direct https URL instead of relying on a redirect - Unified the Linux install command on sh to match the installer script - Updated all CLI and station docs snippets for consistency ### Rename Analytics to Run Analytics URL: https://www.tofupilot.com/changelog/rename-analytics-to-run-analytics The Analytics page is now called Run Analytics, making it clearer that it covers yield, throughput, and duration across your test runs and distinguishing it from Phase Pareto and Process Control. - Renamed the Analytics page to Run Analytics in the dashboard and docs - Added automatic redirects so existing bookmarks and links keep working ### Redesign Measurement Control analytics URL: https://www.tofupilot.com/changelog/redesign-measurement-control-analytics Measurement Control now covers every measurement type and makes it faster to find which measurement is driving failures. - Added support for all measurement types (numeric, boolean, string, multidimensional, and more), each listed and counted in the pareto and detail table - Added a Failures view showing measurement outcomes over time, plus an average Cpk indicator with a period-over-period trend - Reorganized the sidebar with phase, measurement, type, and outcome filters that scope the chart and table together - Improved the table to list every measurement when nothing is selected, with stable columns whether or not a measurement is picked ### Polish Measurement Control charts and filters URL: https://www.tofupilot.com/changelog/polish-measurement-control-charts-and-filters Refinements to the Measurement Control page make the charts, filters, and detail table clearer and more consistent. - Improved the Cpk view with an average Cpk indicator and trend, a capability-based bar color, and a clearer 1.33 threshold marker - Made sidebar filter counts match the chart by using the same default time window, and unified the outcome and value filters with the rest of the sidebar - Improved the measurement table with sortable phase and measurement columns, sample badges, type icons for multidimensional and JSON values, and selection shortcuts to open the related runs or units - Fixed bin deselection in the control chart and tidied the resizable divider between charts ### Filter Measurement Control by run details URL: https://www.tofupilot.com/changelog/filter-measurement-control-by-run-details New sidebar filters let you narrow Measurement Control to specific runs by where and how they were produced. - Added Run ID and Duration filters to scope measurements to specific or longer/shorter runs - Added a Procedure section with Environment (production/preview/development), Deployment (by commit), and Version filters - Each filter shows live counts and narrows the charts and table together ### Filter Phase Pareto by run details URL: https://www.tofupilot.com/changelog/filter-phase-pareto-by-run-details The Phase Pareto page now has the same run-detail filters as Measurement Control. - Added Run ID and Duration filters to scope phases to specific or longer/shorter runs - Added a Procedure section with Environment (production/preview/development), Deployment (by commit), and Version filters - Each filter shows live counts and narrows the chart and table together ### Fix Windows CLI install access denied error URL: https://www.tofupilot.com/changelog/fix-windows-cli-install-access-denied-error Windows users could hit an "Access is denied" error when running the CLI right after installing it, because Windows flagged the downloaded binary as untrusted. - Fixed the installer to clear the Mark-of-the-Web tag on the CLI binary so it runs immediately after install - Removed the need to manually unblock tofupilot.exe before first use ### Grouped filters + deployment filters on tables URL: https://www.tofupilot.com/changelog/grouped-filters-deployment-filters-on-tables Table filter sidebars are now organized into clear sub-groups, and run-detail filters are available across the run-backed pages. - Reorganized the Runs, Logs, Units, API calls, and Batches filter sidebars into grouped sections (Run, Unit, Procedure, Metadata) matching the analytics pages - Added Environment, Deployment, and Version filters to Runs and Logs, plus Run ID and Duration - Exposed previously hidden filters: started/ended/created date ranges on runs, run count and run-mode toggles on units, and a filter sidebar on batches for the first time ### Tidy Runs and Logs filter sidebars URL: https://www.tofupilot.com/changelog/tidy-runs-and-logs-filter-sidebars Small cleanups to the Runs and Logs filter sidebars. - Removed the Run ID filter from the sidebars (filter by run ID via the command palette or search instead) - Renamed the Logs "Run Outcome" filter to "Outcome" ### Logs unit filters + consistent filter names URL: https://www.tofupilot.com/changelog/logs-unit-filters-consistent-filter-names Logs gain the same unit filters as Runs, and filter names are now consistent across every table and analytics sidebar. - Added Part, Revision, Batch, and Sample filters to the Logs sidebar - Standardized date filter labels (Started At, Ended At, Created At, Timestamp) and removed redundant prefixes across Runs, Logs, Units, Batches, Measurement Control, and Phase Pareto ### Connect AI agents to your test data with MCP URL: https://www.tofupilot.com/changelog/connect-ai-agents-to-your-test-data-with-mcp Connect Claude, Cursor, and any MCP client to your TofuPilot data so an AI agent can query and manage your test operations directly, with no API key to copy or rotate. - Added an MCP server you connect by authorizing once in your browser, scoped to a single organization per client - Added read tools for runs, units, parts, batches, procedures, and stations, plus optional write access to create, update, and delete - Added per-connection controls: read-only by default, write opt-in, role-aware access, and a connected-clients list with one-click revoke ### Cleaner MCP connection screen URL: https://www.tofupilot.com/changelog/cleaner-mcp-connection-screen Refined how you connect AI agents to your test data with MCP. - Simplified the authorization screen with a clear read or read-and-write choice per connection - Spelled out exactly what a client can access, including that write includes permanent deletes - Surfaced MCP in the sidebar so it's easier to find and connect a client ### Develop procedures locally, upload when ready URL: https://www.tofupilot.com/changelog/develop-procedures-locally-upload-when-ready Iterate on a procedure in your own working directory and upload runs to the dashboard only when you choose, with no deploy step in the loop. - Added `tofupilot link` to bind a working directory to a dashboard procedure, picking interactively or with `--procedure` for CI. - Added `--upload` to `tofupilot run` so a linked local run syncs to the dashboard, while a plain `run` stays local for fast iteration. - Added `TOFUPILOT_PROCEDURE_ID` to set the target in automation, and `tofupilot unlink` to remove a link. ### Fix Windows install errors on managed devices URL: https://www.tofupilot.com/changelog/fix-windows-install-errors-on-managed-devices Installing the CLI on a managed Windows machine could fail with an unhelpful "Access is denied" and no way to tell why. - Fixed the Windows installer to pinpoint the cause when a blocked launch occurs, with clear next steps and a working alternate install location. - Added a saved install log and step-by-step status so failures are easy to share and resolve. ### Fix Created At filter placement on units page URL: https://www.tofupilot.com/changelog/fix-created-at-filter-placement-on-units-page - Moved the Created At date filter to the top of the Unit filter group - Kept the run-scoped Started At filter in the Run group for clarity ### Faster CLI startup, offline-friendly URL: https://www.tofupilot.com/changelog/faster-cli-startup-offline-friendly The CLI no longer waits on the network at startup when you are already logged in, so commands stay fast offline and on flaky connections. - Throttled the background update check so a burst of commands makes one network call per window instead of one per command. - Made `whoami` cache-first, showing your identity instantly and refreshing only once a day. - Slowed the station update-check cadence to favour stability on production benches. ### Fix headless runs hanging on unit ID URL: https://www.tofupilot.com/changelog/fix-headless-runs-hanging-on-unit-id Headless runs needing manual unit identification could hang forever with no interface to answer the prompt. - Fixed the hang, headless runs now fail fast with clear guidance - Kept dashboard and station operators able to answer identification on upload and deployment runs ### The TofuPilot CLI is now open source URL: https://www.tofupilot.com/changelog/the-tofupilot-cli-is-now-open-source The CLI that runs, deploys, and manages your test procedures is now fully open source under the MIT license, so you can read the code, build it yourself, and contribute. - Published the CLI at github.com/tofupilot/cli with prebuilt binaries for macOS, Linux, and Windows - Added contributor docs, an architecture guide, and a security policy - Unified internal error handling and expanded the test suite for a cleaner, more reliable codebase ### Simplify station deployment status view URL: https://www.tofupilot.com/changelog/simplify-station-deployment-status-view The station page now shows each linked procedure's current deployment at a glance instead of a dense history timeline. - Replaced the per-procedure deployment timeline with a single row showing the latest commit and a Pushed / Not pushed status - Added clear empty states for stations with no linked procedures and procedures with no deployment yet ### CLI now installs and updates in China URL: https://www.tofupilot.com/changelog/cli-now-installs-and-updates-in-china CLI install and update downloads were blocked in mainland China because they were served from GitHub, which is unreachable behind the Great Firewall without a VPN. - Mirrored every CLI release to a global edge network reachable from China - Switched all binary downloads to that mirror so every client uses one fast, China-reachable path - Verified the latest release is reachable in-region so existing installs keep updating without interruption ### Refine station and procedure deployment views URL: https://www.tofupilot.com/changelog/refine-station-and-procedure-deployment-views Cleaner station and procedure pages make it easier to see what is deployed where and to manage deployments. - Improved station deployment items: full-width rows with the commit shown inline and a new "Remove from station" action. - Restyled the procedure page station list to match the deployment view for a consistent look. - Streamlined the station setup tab with clearer framework support (TofuPilot Framework, OpenHTF, pytest, Robot Framework) and a simpler API key empty state. ### Redesigned framework documentation URL: https://www.tofupilot.com/changelog/redesigned-framework-documentation The TofuPilot Framework docs are reorganized into a clearer, deeper reference with interactive previews you can read at a glance. - Added dedicated pages for procedures, phases, measurements, operator UI, plugs, unit identification, execution, logs, attachments, reports, and environments. - Added live previews of measurements and operator UI components, each paired with a text rendering so AI assistants and screen readers get the same information. - Improved the framework home page with a concise overview and direct links to every concept. ### Simplify station login with setup tokens URL: https://www.tofupilot.com/changelog/simplify-station-login-with-setup-tokens Logging in as a station now uses a single, consistent flow. `tofupilot login` authenticates you as a user in the browser, and stations register with a setup token from the dashboard. - Removed the `--station` login flag in favor of setup tokens for every station. - Updated the station login errors to point you straight to the dashboard's setup-token page. - Streamlined the login backend so user and station credentials follow separate, dedicated paths. ### Fix CLI setup for stations in China URL: https://www.tofupilot.com/changelog/fix-cli-setup-for-stations-in-china Test stations in China could not complete first-time CLI setup because a required Python tooling download came from a host that is unreachable there. - Fixed station setup in China by serving the Python environment manager from TofuPilot's own download infrastructure - Added a configurable download source for self-hosted and air-gapped deployments ### See live run progress in the terminal URL: https://www.tofupilot.com/changelog/see-live-run-progress-in-the-terminal Running a procedure with the CLI now shows what's happening as it happens, so a run never finishes silently — and a failure always explains itself. - Added a live console stream of phases, measurements, logs, and the final outcome to every `tofupilot run`. - Surfaced the full error and traceback when a phase or run fails, instead of a bare status. - Defaulted the interactive terminal UI on, and quieted internal plug startup chatter. ### Drill from charts into phases and measurements URL: https://www.tofupilot.com/changelog/drill-from-charts-into-phases-and-measurements Analytics charts now let you drill straight from a high-level view down to the runs, phases, and measurements behind any data point, carrying your filters the whole way. - Added right-click drill-down across the run analytics, phase, and measurement charts so you can jump from runs to phases to individual measurements in a few clicks. - Added clickable outcome tiles and a "Pass Rate" view to the phase and measurement charts, with a Pareto 80% line to surface the vital few. - Improved filter carry-over so the date window, outcomes, parts, batches, and other filters follow you across every drill-down. ### Fix CLI login approval in the browser URL: https://www.tofupilot.com/changelog/fix-cli-login-approval-in-the-browser Approving `tofupilot login` from the browser was failing with a "Device denied" error, blocking new CLI sign-ins. - Fixed the device-approval page so the login code is recognized when you approve it. - `tofupilot login` now completes as expected from both the CLI link and a manually entered code. ### Add full JSON output and clear command feedback URL: https://www.tofupilot.com/changelog/add-full-json-output-and-clear-command-feedback Scripts and agents driving the CLI needed every command to speak JSON, and several commands used to succeed without printing anything. - Added `--json` output to whoami, link, unlink and service status, and made API command errors return structured JSON instead of plain text - Added success messages to all create, update and delete commands (`Created part `) so they no longer succeed silently - Fixed environment bootstrap output corrupting the `--json` event stream during runs ### Set up a station with one login command URL: https://www.tofupilot.com/changelog/set-up-a-station-with-one-login-command Turning a machine into a station no longer needs a separate install step, so the dashboard's one-line setup command fully provisions the station on its own. - Added automatic boot-service setup on station login, so the station restarts after a reboot with no extra command. - Removed the boot service automatically when you log back in as yourself, making login the single command to switch a bench between station and development use. - Removed the separate install command, which is now redundant. ### Filter procedures as you type in the CLI URL: https://www.tofupilot.com/changelog/filter-procedures-as-you-type-in-the-cli The interactive procedure and deployment pickers in `tofupilot link` and `tofupilot run` now filter as you type, so you no longer page through long lists to find one. - Added fuzzy filtering to the link and run pickers — start typing to narrow the list instantly. - Showed a short id next to each procedure name so duplicate names are easy to tell apart. - Fixed `link --procedure ` to report an error and list matching ids when a name is ambiguous, instead of silently picking the first match. ### Document JSON measurements URL: https://www.tofupilot.com/changelog/document-json-measurements JSON measurements let you capture structured objects and arrays such as device state, config snapshots, and diagnostics, and now have their own documentation page. - Added a JSON measurement guide covering how values are captured, how the type is detected, and the validators it supports. - Added a live preview that renders a JSON payload the way it appears in the dashboard. - Clarified that numeric matrices stay multi-dimensional while objects and other arrays become JSON measurements. ### Use any name for your procedure YAML file URL: https://www.tofupilot.com/changelog/use-any-name-for-your-procedure-yaml-file Your procedure file no longer has to be named procedure.yaml. Point your deployment's entry point at any YAML file and TofuPilot runs it. - Added support for custom-named procedure YAML files (any `.yaml`/`.yml`) via the deployment entry point. - Improved repository import so a custom-named procedure is detected and offered in the picker instead of being reported as missing. - Fixed monorepo builds so a sibling procedure with a custom-named YAML ships as source instead of being skipped. ### Add image cards to radio and checklist options URL: https://www.tofupilot.com/changelog/add-image-cards-to-radio-and-checklist-options Radio and checklist components now render as selectable image-card grids when their options include images, replacing the separate image choice and image checklist component types. - Added an `image` field on radio and checklist options that switches rendering to an image-card grid in the operator UI, with `columns`, `aspect`, and `fit` layout controls - Removed the `image_choice` and `image_checklist` component types; procedures should use `radio` and `checklist` with image options instead - Merged the image choice and image checklist documentation into the radio and checklist pages, with redirects from the old URLs ### Add queue get, export and single-entry retry URL: https://www.tofupilot.com/changelog/add-queue-get-export-and-single-entry-retry Stations that run offline queue their runs locally, and operators needed better tools to inspect, recover, and clean up stuck uploads from the CLI. - Added `queue get` and `queue export` to inspect a queued run's failure detail and archive its upload payload before removing it - Added single-entry retry that reports the created run ID or the persisted failure record with `--json`, and exit codes that signal when entries remain queued - Renamed `queue drop` to `queue rm` and added lifecycle status (parked, backoff, attachments pending) plus attempt counts to `queue ls` ### Faster dashboard startup and navigation URL: https://www.tofupilot.com/changelog/faster-dashboard-startup-and-navigation Opening the dashboard previously fired six separate data fetches in sequence just to render the sidebar, and server startup waited on analytics tooling before serving the first request. - Improved sidebar loading by fetching organization, plan, and profile data together in a single round-trip - Reduced server cold-start time by deferring analytics initialization and trimming the server bundle - Reduced the script payload on every page by loading the live-updates client only when it connects ### Export phases and measurements to CSV URL: https://www.tofupilot.com/changelog/export-phases-and-measurements-to-csv Phase Pareto and Measurement Control are the two analytics tables that could not be exported, so getting their data into a spreadsheet meant copying rows by hand. - Added CSV and JSON export to the phase and measurement lists, available from the selection bar once rows are selected. - Added a copy-to-clipboard option so selected rows can be pasted straight into a spreadsheet. - Kept numeric, boolean and text measurement values in separate columns so numbers stay sortable in the exported file. ### Catch YAML typos and fix concurrent run crashes URL: https://www.tofupilot.com/changelog/catch-yaml-typos-and-fix-concurrent-run-crashes Running several CLI processes on one machine is now reliable, and mistakes in procedure YAML surface immediately instead of being silently ignored. - Fixed concurrent runs and station commands interfering with each other through the shared state database: the lock is now held only while in use and contention waits politely instead of interrupting other processes - Added strict procedure YAML validation: unknown fields like `phases:` or `value:` now fail with a clear error naming the field instead of being silently ignored - Fixed orphaned Python worker processes surviving after runs, `tofupilot run file.yml` ignoring the given file, and timeout phases reporting zero duration ### Expand CLI docs with new command references URL: https://www.tofupilot.com/changelog/expand-cli-docs-with-new-command-references Every code example in the CLI and framework docs is now verified to run as written, and the CLI's automation surface is fully documented. - Added reference pages for the imports, logs, phases, measurements and deployments commands - Added a complete `run --json` event-stream reference covering the protocol, every event type, stdin commands and headless CI recipes - Updated every YAML and Python example to the current procedure schema so copy-pasted snippets work out of the box ### Improve scheduled workflow reliability URL: https://www.tofupilot.com/changelog/improve-scheduled-workflow-reliability Scheduled workflows and background maintenance now run on dedicated platform schedules instead of inside request servers, removing a class of intermittent failures. - Fixed intermittent connection errors that could disrupt scheduled workflow runs - Fixed slow responses on station event ingestion caused by background work competing with requests - Improved sign-in cookie handling and pinned strict database connection security ### Speed up procedure list initial load URL: https://www.tofupilot.com/changelog/speed-up-procedure-list-initial-load The procedure list now appears immediately when opening the dashboard, instead of after a blank loading spinner. - Improved initial load by rendering procedure cards, yield circles, and run sparklines directly in the first page response, removing a full client roundtrip - Added skeleton placeholders and kept previous results visible while searching, so the page never flashes empty - Added a "/" keyboard shortcut to focus the search bar on list pages, and create buttons no longer pop in after loading ### Improve run validation and API error messages URL: https://www.tofupilot.com/changelog/improve-run-validation-and-api-error-messages Run uploads now validate timestamps before anything else, and API errors are shorter and clearer. - Added validation rejecting runs and phases whose end time precedes the start time, with a message naming the field - Improved database error messages in the MCP interface to short, readable descriptions ### Speed up the dashboard load (1) URL: https://www.tofupilot.com/changelog/speed-up-the-dashboard-load-1 Data pages now load noticeably faster, starting with the logs page. - Fixed data being fetched twice on page load across logs, API activity, stations, workflows, batches, and measurement analysis pages - Added an instant loading skeleton to the logs page so it no longer waits for data before appearing - Reduced background work during startup so pages become interactive sooner ### Deploy procedures from the CLI URL: https://www.tofupilot.com/changelog/deploy-procedures-from-the-cli Ship a procedure's local source straight from your machine, no git push required. - Added `tofupilot deploy`: packs the linked directory, builds it in the cloud, and streams build logs to your terminal - Added `--prod` to ship to every linked station; preview by default - Streamed build status and logs live until the build finishes ### Show CLI deployments in the dashboard URL: https://www.tofupilot.com/changelog/show-cli-deployments-in-the-dashboard Deployments pushed with `tofupilot deploy` now appear alongside git deployments. - Added source deployments to the deployments list and detail, labeled "Uploaded from CLI" - Showed the source hash in place of a commit for CLI deployments ### Smoother, consistent analytics chart loading URL: https://www.tofupilot.com/changelog/smoother-consistent-analytics-chart-loading Analytics charts now load and render more consistently across the run, phase, and measurement views. - Added uniform loading skeletons that fill the whole chart area, so charts no longer flash a spinner or jump when data arrives. - Fixed pareto charts overflowing past the card edge, which hid bars behind the scrollbar and left an unusable horizontal scroll. ### Speed up the settings page URL: https://www.tofupilot.com/changelog/speed-up-the-settings-page Opening and navigating organization settings is now noticeably lighter and faster. - Removed dozens of unnecessary background requests triggered on every settings visit - Improved how organization logos and avatars load, so they no longer re-download on every visit ### Exact histogram for large numeric measurements URL: https://www.tofupilot.com/changelog/exact-histogram-for-large-numeric-measurements On numeric measurements with more than 500 points, the control chart histogram was built from a sample of the data, so the distribution was approximate and rare out-of-spec values could be missing from the bars while the OOS counter showed them. - Improved the numeric histogram to show the exact distribution over all points, matching the boolean and string charts - Fixed out-of-spec outliers disappearing from the histogram on large datasets; the value range now always covers them - Fixed the histogram tooltip ratio, which could overstate bin proportions on sampled data ### Fix sidebar filters not updating list pages URL: https://www.tofupilot.com/changelog/fix-sidebar-filters-not-updating-list-pages Selecting a sidebar filter on the runs, units, logs, and other list pages updated the URL and the counts but the table kept showing unfiltered rows until a page reload. This was a short-lived regression introduced in the previous daily release. ### Faster list pages and smoother tab switching URL: https://www.tofupilot.com/changelog/faster-list-pages-and-smoother-tab-switching Data pages now show their layout instantly while results load, and the dashboard no longer reloads everything when you return to the tab. - Added instant loading skeletons to units, batches, parts, stations, deployments, commits, and API logs pages so they appear immediately instead of waiting on data - Stopped the dashboard from refetching every open query when you switch back to its browser tab - Sped up the measurement table by removing redundant per-row database work ### Speed up the run analytics page URL: https://www.tofupilot.com/changelog/speed-up-the-run-analytics-page The run analytics page now loads its charts with the data already in place, instead of fetching it twice after the page appeared. - Removed a duplicate data fetch on the default view, so charts render with their numbers immediately on load - Fixed "Open in Table" links so the Runs and Units tables open scoped to the exact window shown in the charts - Loaded part and revision thumbnails in the filters in a single request, removing avatar flicker when changing filters or dates ### Speed up the phase pareto page (2) URL: https://www.tofupilot.com/changelog/speed-up-the-phase-pareto-page-2 The phase pareto page now loads its filters from the same recent window as the chart, instead of scanning a wide history range on first open. - Fixed a cold-load delay where the sidebar filters ran an unbounded query before the page was ready - Loaded the phase, outcome, and sample-class filter counts from the default last-week window, so they appear with the chart instead of after a long wait ### Speed up the phase pareto page (1) URL: https://www.tofupilot.com/changelog/speed-up-the-phase-pareto-page-1 The failures table below the pareto chart now loads with the page instead of fetching separately after it appears. - Loaded the phase failures table from the server so it shows up immediately on first paint - Scoped the table to the same time window as the chart above it, so both show the same period - Stopped sidebar filter counts from loading while the sidebar is collapsed ### Speed up the timeline charts URL: https://www.tofupilot.com/changelog/speed-up-the-timeline-charts The timeline chart above runs, logs, and units tables used to show a blank band while the page loaded, then pop in with a delay. - Added chart bars directly into the initial page response, so the timeline is visible the moment the page appears instead of after a blank pause - Improved loading placeholders to match the chart and table layout exactly, removing visible jumps when content arrives - Fixed charts disappearing into a loading state on every filter change; the previous chart now stays visible while new data loads ### Streamline member offboarding URL: https://www.tofupilot.com/changelog/streamline-member-offboarding Offboarding a team member is now a single action with nothing left to clean up by hand. - Removing a member now automatically deactivates their organization API keys and MCP connections in one step - Improved the remove and re-invite flow so returning members get a fresh start while their test history stays intact - Updated the member management docs to walk through the new offboarding flow ### Speed up the usage page URL: https://www.tofupilot.com/changelog/speed-up-the-usage-page The usage page now loads in well under two seconds instead of over four, so billing and consumption analytics appear almost instantly. - Improved usage analytics query speed by collapsing eight database round trips into two and adding date-range indexes - Added instant rendering from a server-prefetched cache, removing the wait for the page to become interactive before data loads - Improved annual billing views to show a clean monthly breakdown instead of 365 daily bars - Fixed empty monthly and weekly charts for users in time zones ahead of UTC ### Reset filters from empty chart placeholders URL: https://www.tofupilot.com/changelog/reset-filters-from-empty-chart-placeholders When an analytics chart comes up empty because your filters excluded everything, you can now clear them right from the placeholder. - Added a Reset action to the empty state on the run, phase, and measurement analytics charts, so you can recover from an over-filtered view in one click. - The placeholder now tells you when active filters are the reason a chart is empty, and only offers Reset when there's actually a filter to clear. ### Speed up the phase pareto page (3) URL: https://www.tofupilot.com/changelog/speed-up-the-phase-pareto-page-3 Heavy pages now paint faster by streaming less work to the browser and scanning less data on the server. - Rebuilt the timeline chart on list pages (runs, units, logs) to render instantly without a charting library, cutting script work and layout jank - Made the phase pareto page's data queries scan a bounded set of runs instead of the procedure's full history, removing the multi-second wait before the chart appears ### Speed up the dashboard load (2) URL: https://www.tofupilot.com/changelog/speed-up-the-dashboard-load-2 Dashboard pages now paint faster on first load by deferring heavy code that isn't needed until you open a detail panel. - Cut the JavaScript loaded before a page becomes interactive by nearly half - Moved the code viewer and diagram rendering to load on demand instead of on every page - Reduced the analytics script footprint on initial load ### Speed up measurement control links and sidebar URL: https://www.tofupilot.com/changelog/speed-up-measurement-control-links-and-sidebar Opening a measurement control link that already had a measurement selected made every sidebar filter fetch its options separately, even though the data was already loaded with the page. - Improved shared links with a measurement selection: sidebar filters now reuse the data delivered with the page instead of firing six extra requests - Improved internal consistency: measurement control now uses the same first-load data handling as all other list pages, closing the gap that caused the recent sidebar filter bug - Fixed an edge case where certain link parameters unnecessarily skipped the fast first load ### Show procedure images in the operator view URL: https://www.tofupilot.com/changelog/show-procedure-images-in-the-operator-view Reference images defined in your procedure YAML (radio and checklist option images, image components) now load everywhere an operator runs a test, not just in Studio. - Added image resolution to the dashboard operator view and the CLI kiosk so relative image paths from your procedure load correctly - Fixed radio and checklist option image grids that had stopped showing images - Corrected the image component docs to use `default_value` ### Speed up the list pages URL: https://www.tofupilot.com/changelog/speed-up-the-list-pages Clicking a filter used to flash loading placeholders across the whole sidebar, lag the checkbox tick, and swap the table to skeleton rows on every change. - Fixed checkboxes so they respond instantly on click, with options and counts staying visible while updated numbers load in the background - Improved the table to keep showing results during filter changes instead of clearing to a loading state, and selections now reset when filters change - Added smooth open and close animations to filter sections, clear retry messaging when options fail to load, and consistent loading placeholders across all filter types ### Speed up the procedure picker URL: https://www.tofupilot.com/changelog/speed-up-the-procedure-picker The procedure picker now opens instantly instead of waiting on a load when you land on it or open the switcher in the top bar. - Removed the ~680ms load when opening the full-page procedure selector by serving its first results with the page itself - Added background warming of the top-bar procedure switcher so it is ready before you click - Kept search instant and unchanged ### Speed up the phase pareto page (5) URL: https://www.tofupilot.com/changelog/speed-up-the-phase-pareto-page-5 The phase pareto page is now consistently fast for every procedure, including large ones, after finishing the move to a faster data path. - Finished switching the page's queries to the faster lookup so the chart and table load quickly even on procedures with millions of phases - Rolled out automatically in the background with no downtime ### Speed up the phase pareto page (4) URL: https://www.tofupilot.com/changelog/speed-up-the-phase-pareto-page-4 The phase pareto page now opens in well under a second instead of several seconds, matching the other analytics pages. - Reworked how the page loads its data so the chart appears almost instantly - Rolled out in the background with no downtime, switching to the faster version automatically once ready ### Speed up the filter sidebars URL: https://www.tofupilot.com/changelog/speed-up-the-filter-sidebars The filter sidebars on the runs and units pages now load their option counts much faster, even on large organizations. - Sped up how filter option counts are calculated, several times faster on busy pages - Fixed revision number filters missing matches when the number was stored with different capitalization ### Speed up the units page filters URL: https://www.tofupilot.com/changelog/speed-up-the-units-page-filters The units page filter sidebar now loads quickly even when you combine unit filters with run-based ones like outcome or operator. - Extended the faster filter counts to the units page's mixed filter combinations - Kept results identical to before, just faster ### Fix sidebar loading and reset behavior URL: https://www.tofupilot.com/changelog/fix-sidebar-loading-and-reset-behavior Two filter-sidebar fixes on the phase pareto and measurement control pages. - Fixed filters briefly showing "No options available" while loading; they now show a loading skeleton until the options arrive - Fixed the Reset button so it also clears the selected phase or measurement, returning the page to a clean state ### Speed up the phase pareto page (6) URL: https://www.tofupilot.com/changelog/speed-up-the-phase-pareto-page-6 The phase pareto page now loads faster by skipping work most views never need. - Stopped computing the sample-class breakdown on every load; it now loads only when you open the Sample filter, cutting the page's main query time - Sample counts stay live and accurate, updating immediately when a unit's sample type changes ### Add a page loading bar URL: https://www.tofupilot.com/changelog/add-a-page-loading-bar Navigating between pages now shows a thin progress bar at the top, so it's clear when the next page is loading. - Added a slim loading bar that animates across the top during navigation - Matches the light and dark themes ### Show a clear error when a page fails to load URL: https://www.tofupilot.com/changelog/show-a-clear-error-when-a-page-fails-to-load When a list page failed to load, it used to look empty, as if there was no data. It now shows a clear error with a retry button. - Added a proper error state with a "Try again" button to the runs, units, logs, batches, and other list pages - A failed load no longer looks like an empty page ### Fix changelog crash when scrolling for more URL: https://www.tofupilot.com/changelog/fix-changelog-crash-when-scrolling-for-more Scrolling to load more entries on the changelog, news, and guides pages could crash the page instead of fetching the next batch. - Fixed the load-more pagination so infinite scroll reliably loads older entries on changelog, news, and guides. ### Show a real error state on failed list loads URL: https://www.tofupilot.com/changelog/show-a-real-error-state-on-failed-list-loads When a list page failed to load, a network or server error looked identical to having no data — now it's clearly an error you can retry. - Added a dedicated error state with a "Try again" action across runs, units, logs, batches, API activity, and procedures. - Kept existing rows on screen when a "load more" fails, so a failed fetch no longer wipes the list. ### Speed up dashboard load and rendering URL: https://www.tofupilot.com/changelog/speed-up-dashboard-load-and-rendering The dashboard now loads less code up front and re-renders more efficiently, so pages feel faster across the board. - Trimmed the JavaScript loaded on every page, with the heaviest detail pages cut the most - Charts, search, and other heavy panels now load on demand instead of up front - Enabled automatic rendering optimizations for smoother interactions ### Speed up the station page URL: https://www.tofupilot.com/changelog/speed-up-the-station-page The station detail page now loads noticeably less code up front, so it opens faster. - Removed a heavy charting library from the initial load by drawing the telemetry sparklines directly - Loaded the realtime connection only when it's actually needed - Cut about 900KB of JavaScript from the page's first load ### Speed up measurement, chat, and report pages URL: https://www.tofupilot.com/changelog/speed-up-measurement-chat-and-report-pages Charts on the measurement, chat, and report pages now load on demand instead of up front, so the pages open faster. - Deferred chart rendering until it's actually shown - Loaded the code viewer only for measurements that use it - Trimmed the initial load across all three page types ### Trim JavaScript loaded on every page URL: https://www.tofupilot.com/changelog/trim-javascript-loaded-on-every-page Reduced the shared code that every dashboard page loads, so navigation feels lighter across the app. - Shipped only the English text for validation messages instead of every language - Loaded search, notifications, and analytics on demand instead of up front - Cut roughly 300KB from the code shared by every page ### Smoother, more reliable dashboard rendering URL: https://www.tofupilot.com/changelog/smoother-more-reliable-dashboard-rendering Tightened the checks behind the dashboard's automatic rendering optimizations, so the UI stays fast and dependable as it grows. - Strengthened safeguards around how components render and update - Fixed several edge cases surfaced by the new checks - Resolved two subtle display glitches in the process ### Fix measurement control page timeouts URL: https://www.tofupilot.com/changelog/fix-measurement-control-page-timeouts The measurement control page could time out on procedures with millions of measurements, leaving the Pareto charts unable to load. - Fixed the measurement control page timing out on large procedures, so the Pareto and Cpk charts load reliably. - Improved how historical measurement data is aggregated in the background, keeping the page fast as your data grows. - Fixed the live runs toggle getting stuck on "Reconnecting" after a connection drop. ### Fix uneven gaps in timeline chart bars URL: https://www.tofupilot.com/changelog/fix-uneven-gaps-in-timeline-chart-bars The activity timeline above each list now renders with consistent spacing between bars. - Fixed uneven horizontal gaps between bars in the timeline chart shown on the runs, units, logs, and API activity pages. - Aligned the chart's spacing with its loading skeleton so the layout no longer shifts as data loads. ### Speed up public content pages URL: https://www.tofupilot.com/changelog/speed-up-public-content-pages Changelog, news, guides, templates, and roadmap detail pages now load faster on first view. - Cached each public detail page behind a single shared, tagged getter, cutting roughly four uncached database round-trips per view down to one. - Busted the cache automatically on publish so updates still appear immediately. ### Speed up homepage hero image load URL: https://www.tofupilot.com/changelog/speed-up-homepage-hero-image-load The homepage hero image was loading at low priority and arriving ~1.4s into page load, hurting perceived speed. - Fixed the high-priority hint on the theme-aware hero image so it now downloads first. - Marked the visible hero image as eager so the browser stops deferring the largest content element, dropping load to under 800ms. ### Document Python version selection URL: https://www.tofupilot.com/changelog/document-python-version-selection Clarifies how TofuPilot picks the Python version for a procedure so users avoid "No interpreter found" and package-incompatibility errors. - Added a Python version section explaining selection happens at the minor level (3.12), not the patch (3.12.4), with supported minors 3.10 to 3.14. - Documented the gotcha for packages limited to older minors and how to cap the upper bound in requires-python. - Noted that air-gapped stations can only use Python minors already installed locally, since provisioning a new minor requires fetching the interpreter. ### Show attachment images in the operator UI URL: https://www.tofupilot.com/changelog/show-attachment-images-in-the-operator-ui The CLI kiosk and the dashboard operator UI now support attachment images. Attach a board photo, probe shot, or failure close-up during a run and it shows inline next to the phase instead of as a bare filename, so the operator can compare it against the unit while the run is live. - Added inline image rendering for run attachments in the CLI kiosk (served locally) and the dashboard operator UI (served after upload) - Fixed station runs so attachment images reach the dashboard, matching OpenHTF and Robot runs - Improved deployment history to show who ran a CLI deploy instead of "unknown" ### Run stations as root with launch on boot URL: https://www.tofupilot.com/changelog/run-stations-as-root-with-launch-on-boot Test stations that need root access to their hardware can now start automatically at boot. Previously enabling launch on boot as root failed because the station installed a user-level service. - Added automatic system-service install when the station runs as root, so launch on boot works without a login session. - Hid the on-screen kiosk option for root stations, which have no display, and made clear that these stations are controlled from the dashboard. - Showed a clear, actionable message instead of a raw systemd error when launch on boot cannot be enabled. ### Deploy on any Python from 3.10 to 3.14 URL: https://www.tofupilot.com/changelog/deploy-on-any-python-from-310-to-314 Deployments now build on the Python version your procedure declares, instead of always on 3.12. A procedure pinning `requires-python = ">=3.13"` previously failed to build with "No interpreter found for Python 3.13". - Added support for building procedures on Python 3.10 through 3.14, selected from your `pyproject.toml` `requires-python`. - Improved error messages when a version can't be satisfied, pointing you to a supported range instead of failing cryptically. - Kept the default at 3.12 when `requires-python` is omitted, so existing procedures build unchanged. ### Support pre-baked answers for OpenHTF prompts URL: https://www.tofupilot.com/changelog/support-pre-baked-answers-for-openhtf-prompts Headless and automated OpenHTF runs can now answer operator prompts ahead of time, so a prompt no longer stalls an unattended run. - Fixed operator prompts ignoring pre-baked input values during automated runs, which caused them to wait and eventually time out. - Added automatic prompt resolution when every required input is supplied up front, matching the behavior of native procedures. ### Fix guide and template pages failing to load URL: https://www.tofupilot.com/changelog/fix-guide-and-template-pages-failing-to-load Guide and template pages were briefly returning an error instead of loading. - Fixed an issue that prevented guide and template pages from opening - Restored these pages so they load correctly again ### Fix update failures from network blips URL: https://www.tofupilot.com/changelog/fix-update-failures-from-network-blips A transient connection drop right after a network switch or wake-from-sleep could abort `tofupilot update` even when the server was healthy, and an immediate re-run would succeed. - Fixed `tofupilot update` failing on a single transient network error when checking for the latest version - Added automatic retries with backoff so a brief connection hiccup no longer interrupts the update ### Fix attachment upload from the CLI URL: https://www.tofupilot.com/changelog/fix-attachment-upload-from-the-cli Attachments produced during a CLI run now upload reliably and display correctly on the dashboard. - Fixed `attach.data` attachments being silently dropped on native test runs so they never reached the dashboard. - Fixed `attach.file` attachments uploading without a size, type, or image preview. - Improved cleanup so attachment files left by a failed or aborted run no longer pile up on disk. ### Retry interrupted update downloads URL: https://www.tofupilot.com/changelog/retry-interrupted-update-downloads A connection drop during the binary download — common after a network switch or wake-from-sleep — could abort `tofupilot update` partway through, even when the server was healthy. - Fixed `tofupilot update` failing when the connection dropped mid-download - Added automatic retries with backoff so an interrupted transfer restarts cleanly instead of aborting the update ### Fix kiosk operator UI on root test stations URL: https://www.tofupilot.com/changelog/fix-kiosk-operator-ui-on-root-test-stations Headless test jigs that run as root could no longer open the kiosk operator UI, leaving runs stuck on unit identification. - Fixed the kiosk operator UI failing to start for foreground `run --kiosk` on root stations. - Restored remote viewing of the kiosk over an SSH port-forward on headless machines. - Kept the root station service hardened: its command channel still refuses to bind as root. ### Fix station logout when deploying as a user URL: https://www.tofupilot.com/changelog/fix-station-logout-when-deploying-as-a-user Running `tofupilot deploy` on a station required a user login, which previously overwrote the station's own credentials and stopped its service from starting. - Fixed user and station logins so they no longer overwrite each other; a station stays authenticated after you log in as a user to deploy. - Improved the station service so it keeps running after a user login on the same machine. - Updated existing stations to migrate automatically on upgrade, with no re-login required. ### Use system Python packages in deployments URL: https://www.tofupilot.com/changelog/use-system-python-packages-in-deployments Procedures can now use Python packages installed on the station that pip can't install, like native instrument drivers and vendor SDKs. - Added a "System packages" toggle on the station Setup page that lets deployment environments reach the machine's system Python packages. - Kept deployments isolated by default; the setting is opt-in per station and applies on the next deploy. ### Fix operator answers not saving from terminal URL: https://www.tofupilot.com/changelog/fix-operator-answers-not-saving-from-terminal Operator answers to radio, dropdown, and text prompts now record correctly no matter how a run is launched, fixing a crash when a later phase read a measurement from an earlier prompt. - Fixed bound measurements not being saved on terminal (`tofupilot run`) and automated `--json` runs, which previously worked only in the kiosk and Studio. - Fixed text prompts skipping length and pattern validation on automated runs, so invalid values are now rejected consistently across every surface. - Improved prompts to record their default value when left unanswered, matching the kiosk. ### Fix duplicate deployments from GitLab pushes URL: https://www.tofupilot.com/changelog/fix-duplicate-deployments-from-gitlab-pushes A single GitLab push could appear as several identical deployments when a repository had accumulated more than one webhook. - Fixed GitLab webhook registration so each repository keeps exactly one hook, removing stale duplicates automatically. - Added a safeguard that ignores repeated deliveries of the same push, so one commit produces one deployment per procedure. ### Fix desktop shortcut on localized systems URL: https://www.tofupilot.com/changelog/fix-desktop-shortcut-on-localized-systems Installing the station desktop launcher on a non-English system put the shortcut in the wrong place, so it never appeared for the operator. - Fixed the launcher to use the system's localized Desktop folder (such as Bureau on French installs) instead of always creating an English "Desktop" folder. - Aligned shortcut install, detection, and removal across Linux, Windows, and macOS so they all target the same folder, including OneDrive-redirected desktops on Windows. ### Speed up dashboard with infrastructure upgrade URL: https://www.tofupilot.com/changelog/speed-up-dashboard-with-infrastructure-upgrade We've improved our backend infrastructure to make the dashboard faster and more reliable. - Improved database response times by running compute closer to the data layer - Fixed a class of transient connection errors that could occasionally cause page-load failures - Reduced latency for dashboard requests ### Rename Cycle Time metric to Average Duration URL: https://www.tofupilot.com/changelog/rename-cycle-time-metric-to-average-duration The analytics metric labeled "Cycle Time" actually measures average per-run test duration, not the manufacturing inter-completion cadence the term implies. Renamed it to remove the ambiguity for test and process engineers. - Renamed the "Cycle Time" KPI card, duration chart tooltip, and phase analysis headline to "Average Duration" - Clarified that the value reflects average passing-run execution time, not unit-to-unit cadence ### Speed up list pages and refine the timeline URL: https://www.tofupilot.com/changelog/speed-up-list-pages-and-refine-the-timeline The runs, units, logs, and API activity pages now paint their timeline chart instantly while keeping the richer interactive look. - Improved first paint on list pages by loading the interactive chart layer off the critical path, so the bars appear immediately. - Restored the smoother timeline look with hover tooltips and drag-to-filter range selection. - Evened out the table header placeholders so columns line up while a page loads. ### Add MCP icon and group it under Integrations URL: https://www.tofupilot.com/changelog/add-mcp-icon-and-group-it-under-integrations The MCP server now lives alongside your other connections for easier discovery. - Added the MCP logo to the settings sidebar and connected-client list - Moved MCP into the Integrations section next to GitHub, GitLab, and Linear ### Sign and notarize macOS CLI builds URL: https://www.tofupilot.com/changelog/sign-and-notarize-macos-cli-builds macOS now runs the TofuPilot CLI without Gatekeeper blocking it, so downloaded builds launch without the "cannot be opened because the developer cannot be verified" warning. - Signed the macOS CLI binary for both Apple Silicon and Intel with a new Apple Developer ID certificate - Notarized every build with Apple so it passes Gatekeeper checks on first run ### Make filters harder to break URL: https://www.tofupilot.com/changelog/make-filters-harder-to-break Filters that silently matched nothing were hard to notice and easy to reintroduce. Alongside the fix, we tightened how filters are built and tested. - Rebuilt the filter search syntax so quoted values survive intact, covered by an exhaustive test suite - Made a misconfigured filter fail immediately instead of quietly rendering nothing - Fixed long filter names pushing their counts out of column, and shortened them with the full name on hover ### Fix stations dropping offline mid-run URL: https://www.tofupilot.com/changelog/fix-stations-dropping-offline-mid-run A single oversized event — a large image in a prompt, a verbose log dump or a long measurement trace — used to cut a station's live link for the rest of the session, so runs kept passing locally while the dashboard showed the station as offline. - Fixed the live link dropping when a phase sends an unusually large payload: the event is now shrunk to fit instead of breaking the connection. - Added readable stand-ins for degraded content — an oversized image becomes a short note explaining the operator screen still shows the full picture, and truncated logs keep their start and end with an explicit marker. - Kept normal runs untouched: only the payloads that would break the connection are degraded, and phase outcomes always arrive. ### Add a red STOP button to abort a run URL: https://www.tofupilot.com/changelog/add-a-red-stop-button-to-abort-a-run Operators could not find how to abort a running sequence: the stop control was a small grey icon that read as decoration rather than an emergency control. - Added a solid red STOP button that morphs into KILL after a stop, with a fixed width so the bottom bar never reflows. - Added a 500 ms delay before KILL becomes available, so a touchscreen double-tap cannot escalate a graceful stop into a forced kill. - Fixed the button keeping the previous run's state after a restart or a lost command, and made a kill re-sendable if the run has not ended after 5 seconds. ### Run a specific Python file with tofupilot run URL: https://www.tofupilot.com/changelog/run-a-specific-python-file-with-tofupilot-run Projects with several test stages sharing one codebase could not validate a stage locally: local runs always fell back to main.py, so the only workarounds were renaming files or keeping separate checkouts. - Added support for naming the entry file directly, so `tofupilot run ./stage1_entry.py` runs that stage instead of main.py, with a warning when the framework expects a directory rather than a file. - Fixed ID columns in `tofupilot ls` being cut to eight characters, so IDs can now be copy-pasted straight into get, update, rm and link. - Improved the logged-out and "no procedure found" messages to show the direct-file form, annotated as needing no account. ### Continue between phases with the keyboard URL: https://www.tofupilot.com/changelog/continue-between-phases-with-the-keyboard Operators work with their hands on a barcode scanner, not a mouse, but only phases starting with a text input kept the keyboard alive — an acknowledge prompt, a switch, a radio group or a bare Continue step forced a reach for the mouse once per phase. - Added keyboard focus on every prompt shape, so Enter — including the trailing Enter a barcode scanner sends — acknowledges a phase and moves to the next one. - Added Cmd/Ctrl+Enter to confirm a prompt from anywhere, including inside a multi-line text field where Enter must stay a newline. - Fixed a held Enter skipping a prompt the operator never saw, and restored focus to the form automatically after a failed send so a retry needs no mouse. - Added a ⏎ hint on the Continue button, shown only when Enter really confirms. ### Fix Logs sidebar filters not applying URL: https://www.tofupilot.com/changelog/fix-logs-sidebar-filters-not-applying Clicking a filter in the Logs sidebar now actually filters the list. - Fixed the Level, Source, Outcome, Station, User, and Serial Number sidebar filters doing nothing when clicked - Restored parity so sidebar filters behave the same as the search-bar filters ### Fix long commit text overflowing deploy modal URL: https://www.tofupilot.com/changelog/fix-long-commit-text-overflowing-deploy-modal Long commit messages and procedure names no longer break the manual deployment dialog layout. - Fixed the commit and procedure fields overflowing past the right edge of the New Manual Deployment modal - Improved long text to truncate cleanly with an ellipsis while keeping the commit hash and icons visible ### Launch a public questions and answers hub URL: https://www.tofupilot.com/changelog/launch-a-public-questions-and-answers-hub Common questions about TofuPilot now have a public home: browse direct answers by category or share a link to a single question. - Added a questions hub at tofupilot.com/questions with category filters and instant search - Added a dedicated page per question with related questions, so every answer has its own shareable link - Improved search visibility with structured data and sitemap coverage, in English and French ### Choose which orgs and access each AI agent gets URL: https://www.tofupilot.com/changelog/choose-which-orgs-and-access-each-ai-agent-gets Connecting an AI agent through MCP now lets you pick exactly which organizations it can reach and whether each allows write. - Added a multi-select organization picker to the MCP consent screen, with a per-organization read or write toggle - Added org switching so an agent moves between your authorized organizations, with write access following your role in each - Improved revoke to disconnect a client across every organization at once, not just the current one ### Manage teams and clean up data from AI agents URL: https://www.tofupilot.com/changelog/manage-teams-and-clean-up-data-from-ai-agents MCP now exposes more of your workspace to AI agents, so they can manage teams and remove test data the same way you do in the dashboard. - Added team tools: list teams and members, create and rename teams, add or remove members, and delete a team - Added delete tools for runs and units, plus tools to add or remove units from a batch ### Fix API list crash when called with no filters URL: https://www.tofupilot.com/changelog/fix-api-list-crash-when-called-with-no-filters A plain runs or units list call from any SDK returned a 400 error because the optional metadata filter was rejected when empty. Basic list calls now work again. - Fixed runs.list and units.list returning "expected record, received string" when called without a metadata filter - Fixed metadata filters not applying over the API when passed as a query parameter ### Fix Microsoft SSO sign-up with profile photo URL: https://www.tofupilot.com/changelog/fix-microsoft-sso-sign-up-with-profile-photo New users signing in through Microsoft or Azure AD SSO could fail to be created when their account had a profile photo, because Microsoft sends the photo as inline image data that exceeded an internal limit on the avatar field. - Fixed account creation for Microsoft and Azure AD SSO users who have a profile photo - Improved handling of inline profile photos so they no longer block sign-up - Added a fallback to initials when no uploaded avatar is set ### Fix filters on names containing spaces URL: https://www.tofupilot.com/changelog/fix-filters-on-names-containing-spaces Filtering on a name that contains a space — a phase called "Contact Voltage Drop", a batch called "Lot 2024 rev B" — matched only the first word and returned nothing. - Fixed filters on values containing spaces or commas, everywhere filters are used - Added export to the Phase Pareto and Measurement Control toolbars, matching runs and units - Added the search bar to Measurement Control ### Fix operator UI examples the CLI rejected URL: https://www.tofupilot.com/changelog/fix-operator-ui-examples-the-cli-rejected The Text and Progress pages documented a `value` field that the CLI does not accept, so every example on those pages failed to load when copied into a procedure. - Fixed the Text and Progress operator UI pages to use `default_value`, the field the CLI actually reads, in all eleven YAML examples and both property tables. ### Improve API reliability under low-traffic load URL: https://www.tofupilot.com/changelog/improve-api-reliability-under-low-traffic-load Intermittent 60-second delays could affect API requests and station status checks during low-traffic periods, caused by background processing competing with database connections as infrastructure scaled down between requests. - Fixed background processing after run creation so it completes reliably without delaying or blocking subsequent requests - Improved database connection handling for internal job processing, eliminating connection errors during idle-to-active transitions - Added a 20-second limit on background database operations so a slow query can no longer stall a request slot ### Run OpenHTF 1.3.0 through 1.6.1 URL: https://www.tofupilot.com/changelog/run-openhtf-130-through-161 Tests running an older pinned OpenHTF could not use the CLI at all. Every release from 1.3.0 onward now works the same way. - Added support for OpenHTF 1.3.0 through 1.6.1, with the same measurements, validators, attachments and phase outcomes on every release. - Fixed attachments being dropped on releases below 1.5.2. ### Show CLI deployments without a linked repo URL: https://www.tofupilot.com/changelog/show-cli-deployments-without-a-linked-repo Deploying with the CLI created a deployment you could open by link, but the Deployments page still claimed no repository was connected and listed nothing. - Fixed the Deployments page hiding CLI deployments on procedures with no linked Git repository. - Fixed the same blank state on the organization-wide Deployments page. - Updated the empty state to mention `tofupilot deploy` alongside connecting a repository. ### Fix new procedures missing from list URL: https://www.tofupilot.com/changelog/fix-new-procedures-missing-from-list Newly created procedures now appear in the procedure list right away, without needing a page refresh. - Fixed the procedure list not showing procedures you just created until a manual refresh - Ensured the list and switcher both update immediately after creating a procedure ### Keep procedure lists fresh after changes URL: https://www.tofupilot.com/changelog/keep-procedure-lists-fresh-after-changes Creating or renaming a procedure now updates every place it appears, without needing a page refresh. - Fixed procedures created through the import and clone setup flows not appearing in the list until a refresh - Fixed a renamed procedure showing its old name on the dashboard cards, the procedure switcher, and the stations table - Updated the units filter sidebar to reflect newly created and renamed procedures right away ### Use one MCP connection for all your orgs URL: https://www.tofupilot.com/changelog/use-one-mcp-connection-for-all-your-orgs Connect your AI agent once and reach every organization you belong to, switching between them mid-session instead of re-authorizing for each one. - Added `list_orgs` and `set_org` tools so an agent can see your organizations and switch the active one on the fly - Fixed the connected-clients list leaking connections across organizations - Fixed a reconnect loop that blocked members of more than one organization - Improved the sign-in callback so the browser no longer shows a connection error after you authorize ### Accept invitations from the org switcher URL: https://www.tofupilot.com/changelog/accept-invitations-from-the-org-switcher Switching between organizations is now easier, and invitations you haven't accepted yet show up where you'd expect them. - Added pending invitations to the organization switcher, so you can accept an invite without hunting for the email - Improved the expired-demo experience with a dedicated screen that keeps the switcher available so you can move to your other workspaces ### Invite members from AI agents, safer roles URL: https://www.tofupilot.com/changelog/invite-members-from-ai-agents-safer-roles You can now invite teammates to your organization directly from connected MCP, and invites can no longer grant a role above your own. - Added a tool to invite members by email from connected AI assistants, with the invite sent automatically. - Prevented inviting or promoting someone to a role higher than yours, closing a privilege-escalation gap across the dashboard and API. - Filtered the invite role picker to the roles you're allowed to grant. ### Clearer procedure IDs in docs and SDK examples URL: https://www.tofupilot.com/changelog/clearer-procedure-ids-in-docs-and-sdk-examples Code examples now use an obviously-fake placeholder procedure ID instead of a name or a real-looking UUID, so a copy-pasted snippet can't accidentally target a real procedure and clearly reads as a value to replace. - Replaced name-style (`FVT1`) and a real-looking UUID with `xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx` across the OpenHTF and SDK docs and the V1 API example. - Clarified that `procedure_id` is the dashboard procedure UUID (a legacy external identifier only resolves if one was set). - Updated the OpenHTF client docstrings and released python client 2.13.2. ### Deploy and run behind a strict firewall URL: https://www.tofupilot.com/changelog/deploy-and-run-behind-a-strict-firewall Stations on locked-down networks could not reach the third-party hosts used to download the Python runtime and to transfer deployment archives, so setup failed where IT could not allowlist them. - Added a TofuPilot-hosted mirror for Python interpreter downloads, so station networks now only need `*.tofupilot.app`, `tofupilot.sh` and `dl.tofupilot.sh` (plus PyPI). - Moved deployment uploads and downloads to `artifacts.tofupilot.app`, with no change to how stations work — they follow whatever address the API returns. - Added a clear error when a project's sources exceed the 95 MB upload limit, instead of an opaque failure part-way through. - Added overrides for air-gapped setups: point the mirror at your own host, or turn it off to use the upstream source. ### Strengthen user management on self-hosted URL: https://www.tofupilot.com/changelog/strengthen-user-management-on-self-hosted User seat limits are now enforced server-side on self-hosted instances, so member management respects your license. - Enforced the user cap on invitations, matching how station limits already work. - Kept re-invites and role changes unaffected, since they add no seat. ### Improve self-hosted deployment migration URL: https://www.tofupilot.com/changelog/improve-self-hosted-deployment-migration Self-hosted instances migrating to the new deployment architecture now get a smoother, more reliable experience, even on restricted networks. - Improved resilience when optional deployment services are not yet configured, removing recurring errors from instance logs. - Added a prebuilt build environment so new deployments start faster without downloading dependencies at boot. - Fixed sign-in for existing members using single sign-on after upgrading from older versions. ### Fix rare sign-in lockout after security upgrade URL: https://www.tofupilot.com/changelog/fix-rare-sign-in-lockout-after-security-upgrade A recent security upgrade hardened how sign-ins from identity providers attach to existing accounts, which could lock out a small set of long-standing accounts on upgraded self-hosted instances. ### Rename Teams to Access Groups + free for all URL: https://www.tofupilot.com/changelog/rename-teams-to-access-groups-free-for-all Teams are now called Access Groups and available to every organization, no add-on required. - Renamed Teams to Access Groups across the dashboard, docs, and pricing - Removed the add-on requirement so any organization can scope station and member access - Updated documentation with the new Access Groups guide ### Fix automatic database backups before upgrades URL: https://www.tofupilot.com/changelog/fix-automatic-database-backups-before-upgrades Self-hosted instances take a database snapshot before applying upgrade migrations, but a version mismatch had silently prevented it from completing, and snapshots were not kept across upgrades. - Fixed the pre-upgrade database backup so it completes and persists in a dedicated storage volume across upgrades - Added clear log output when a backup fails, with an optional strict mode (REQUIRE_BACKUP) that halts the upgrade instead of continuing without one - Limited backups to one per version with automatic cleanup so they never fill the server disk ### Fix invitation acceptance for SSO users URL: https://www.tofupilot.com/changelog/fix-invitation-acceptance-for-sso-users New team members signing in through Microsoft SSO could hit an "Invalid invitation" error when opening their invite link. - Fixed invitation acceptance for users whose identity provider does not report a verified email - Improved invitation error pages to show the actual reason instead of a generic message ### Configure plugs from your procedure file URL: https://www.tofupilot.com/changelog/configure-plugs-from-your-procedure-file Plugs can now receive settings like instrument addresses directly from your procedure file, so you no longer have to hard-code them in Python or duplicate a plug class per instrument. - Added an optional config mapping on each plug that is passed to the plug class as keyword arguments - Added support for running several instruments of the same model by giving each plug its own config - Documented the new config option and the multi-instrument pattern in the plugs guide ### Run and unit metadata added to the framework URL: https://www.tofupilot.com/changelog/run-and-unit-metadata-added-to-the-framework High-value units are often tested while undergoing modifications, and standard identifiers like revision number can't capture that state — custom metadata now records it on every run and unit. - Added `run.metadata` and `unit.metadata` to the Python phase API, validated at assignment with clear errors and included even when a phase fails - Added operator-prompted unit metadata fields to the procedure file, rendered in the unit identification form next to the serial number - Added metadata to uploaded reports and Studio local reports so runs and units can be filtered by any key ### Connect Bitbucket Data Center repositories URL: https://www.tofupilot.com/changelog/connect-bitbucket-data-center-repositories Teams hosting their code on a self-managed Bitbucket Data Center instance can now link it to TofuPilot, alongside GitHub and GitLab. - Added Bitbucket Data Center as a git provider, connected with your instance URL and an HTTP access token - Added automatic repository sync and webhooks so procedures deploy on push, with commit details shown in deploy views - Added a Bitbucket settings page to manage the connection and replace tokens ### Debug Python phases with a debugger URL: https://www.tofupilot.com/changelog/debug-python-phases-with-a-debugger Debugging a failing phase used to mean adding print statements; you can now attach VS Code or any debugpy client and step through your Python phase code with breakpoints. - Added a debug mode to the CLI (`tofupilot run --debug`) that starts a debugger listener, runs a single worker, and disables phase timeouts so a breakpoint pause isn't cut off - Added a Debugging guide covering debug mode and manual debugpy attachment, with a ready-to-use VS Code configuration ### Document procedure entry points and run_if URL: https://www.tofupilot.com/changelog/document-procedure-entry-points-and-run-if Teams that run several procedures from one codebase no longer have to rename files to main.py or guess which file a station executes. - Added a full reference for the procedure entry point and root directory settings: accepted values, defaults for each framework, when a change takes effect, and how imports resolve for entry points in subdirectories. - Added a monorepo guide for deploying several procedures from one repository, including procedure files with custom names. - Documented how to run a subset of phases with OpenHTF's run_if option, and how skipped phases appear on the dashboard. ### Fix phases timing out at 60s during plug calls URL: https://www.tofupilot.com/changelog/fix-phases-timing-out-at-60s-during-plug-calls Long-running hardware operations — thermal soaks, firmware flashes, slow measurements — were killed after exactly one minute, even when the phase allowed more time. - Fixed plug method calls dying at 60 seconds: they now run for as long as their phase allows, with no time limit when the phase has none - Improved timeout errors to name the interrupted plug method instead of a bare "timed out" - Documented the phase timeout field: duration syntax, bounds, and how the deadline covers plug calls and operator prompts ### Fix self-hosted license creation in Orbit URL: https://www.tofupilot.com/changelog/fix-self-hosted-license-creation-in-orbit Requesting a self-hosted license in Orbit failed with an unauthorized error for every account, blocking new self-hosting evaluations. - Fixed license creation by resolving your organization from your membership instead of a session flag that was never set - Added clear error messages for each failure case: signed out, no organization yet, missing admin role, or multiple organizations - Fixed stale changelog links from search results to land on the changelog index instead of a 404 page ### Fix DNS failures on linux stations URL: https://www.tofupilot.com/changelog/fix-dns-failures-on-linux-stations On some networks, the linux CLI failed to connect with a DNS "Try again" error while other tools on the same machine resolved fine, because the static linux build shipped a strict built-in resolver. - Fixed the linux build to use a more robust DNS resolver that tolerates unanswered parallel queries and falls back to TCP - Kept the system resolver on macOS and Windows, which were not affected ### Add custom CA support to the Rust SDK URL: https://www.tofupilot.com/changelog/add-custom-ca-support-to-the-rust-sdk The Rust SDK can now connect to self-hosted instances that use a private certificate authority. - Added `add_root_certificate` and `root_certificate_from_pem_file` to the client configuration. - Switched to the system trust store by default, so system-installed certificates work without code changes. - Updated the self-hosting guide with the Rust certificate setup. ### Clarify certificate setup for self-hosted URL: https://www.tofupilot.com/changelog/clarify-certificate-setup-for-self-hosted Self-hosted instances often run behind a private CA, and the deploy guide only showed one client's certificate setup while still documenting the retired v1 Python client. - Removed the legacy Python SDK v1 examples from the self-hosting deploy guide, leaving OpenHTF and the current Python SDK. - Added certificate guidance for every client: the Python SDK accepts a custom certificate directly, while OpenHTF and the C++ and MATLAB SDKs use the standard OpenSSL environment variable. - Documented that the Rust SDK cannot connect to an instance using a private certificate authority yet. ### Fix missing charts for imported test history URL: https://www.tofupilot.com/changelog/fix-missing-charts-for-imported-test-history Runs uploaded with an older start date, such as imported history or a migration from another system, were missing from the Measurement Control charts and filters even though the measurement table below listed them. - Fixed backdated runs not appearing in measurement analytics: charts, trends and sidebar filters now include them as soon as the data is uploaded - Improved how historical data is folded into the pre-computed analytics so charts stay fast once the import settles ### Fix cut-off labels on the measurement chart URL: https://www.tofupilot.com/changelog/fix-cut-off-labels-on-the-measurement-chart The Y axis on a measurement control chart could cut the leading digits off its own labels, so the scale read as non-linear and the same number appeared twice at two different heights. - Fixed axis labels being clipped, which hid the most significant digits of every long value and made limit lines look misplaced - Rounded the axis values so they stay short and readable whatever the measurement scale - Kept enough decimals on narrow ranges, where every label used to collapse to 0 ### Fix stuck station runs with a clear error URL: https://www.tofupilot.com/changelog/fix-stuck-station-runs-with-a-clear-error A station run could get stuck on a spinner with no explanation when the test process was blocked before it could start — often by antivirus or endpoint-protection software holding the newly launched process on a locked-down machine. - Fixed runs hanging indefinitely: the station now surfaces a clear, actionable error instead of an endless spinner when a run cannot start. - Added a "still starting" hint in the operator UI that points you to the terminal if a run is slow to begin. - Improved crash messages so a failed run tells you whether it hit a Python error or was stopped from outside. ### Stop framework runs hanging silently at start URL: https://www.tofupilot.com/changelog/stop-framework-runs-hanging-silently-at-start A station run of a framework procedure could sit on the spinner forever with no error when the test worker process was blocked from starting — most often by security software on locked-down machines. - Fixed framework runs hanging indefinitely at startup: the station now stops waiting after a deadline and shows a clear, actionable error naming the likely cause. - Added the worker's own error output to the failure message, so a crash during startup points straight at the offending line. ### Export every result matching your filters URL: https://www.tofupilot.com/changelog/export-every-result-matching-your-filters Exporting from Phase Pareto or Measurement Control only covered the rows already loaded on screen — filter to 295 phase executions, export, get 50. - Added "Export all matching" to both export menus — the file now contains every row matching your filters, with a clear warning if the export is very large - Added selection chips to the Measurement Control search bar, so the selected measurement shows up there and can be cleared in one click - Fixed removing a filter chip doing nothing, and filters occasionally conflicting when changed quickly from different places - Made Measurement Control load faster on first visit ### Fail fast when security software blocks a run URL: https://www.tofupilot.com/changelog/fail-fast-when-security-software-blocks-a-run On locked-down stations, endpoint-protection software could suspend the Python worker or block the venv interpreter, leaving runs hanging forever with no explanation. - Added startup watchdogs that detect a stalled Python worker (30s to respond, 300s to finish importing) and fail the phase with clear guidance to allowlist the venv in EDR/antivirus software - Fixed `tofupilot pull` to name AppLocker/WDAC/EDR blocking as the cause instead of a bare "Access is denied" error, with one automatic retry for transient scans - Added a per-run event log at `~/.tofupilot/logs/` referenced from stall diagnostics for faster troubleshooting with IT ### Fix selections reverting after quick clicks URL: https://www.tofupilot.com/changelog/fix-selections-reverting-after-quick-clicks Changing a measurement selection quickly — select, deselect, select again — could end up in the wrong state a few seconds later, showing no results until the page was reloaded. - Fixed rapid filter and selection changes occasionally reverting to an earlier state (a race in the URL state library, fixed upstream and upgraded) - Fixed measurement names containing a comma breaking multi-selection entirely - Selecting a measurement now applies in a single step instead of two, removing a flicker ### Keep instruments connected between runs URL: https://www.tofupilot.com/changelog/keep-instruments-connected-between-runs Reconnecting to instruments between units costs time on every run — slow VISA sessions, TCP links, or serial handshakes were re-established for each unit tested on a station. - Added `scope: station` for plugs: the connection opens once, is shared by every run on the station, and closes when the station stops - Health-checked the held instance before each run, respawning it if the process died, the plug definition changed, or a new deployment was applied - Renamed plug and phase scopes to `slot` / `execution` / `station`; the legacy `each` / `all` spellings still parse ### Get alerted to quality incidents early URL: https://www.tofupilot.com/changelog/get-alerted-to-quality-incidents-early TofuPilot now watches your production test data and tells you when quality slips — automatically, with no dashboards to babysit. - Added automatic detection for measurement drift, yield drops, and streaks of failed runs — each judged against the procedure's own history, with nothing to configure. - Added checks for units that get retested too often and for reference (golden) samples that stop behaving — early signs of a drifting fixture or test. - Added custom alert rules with your own thresholds and scope, plus an in-app inbox and email notifications when something fires. - Added full Alerts documentation: alert types, automatic detection, rules, and the alert lifecycle. ### Sign in with your Atlassian account URL: https://www.tofupilot.com/changelog/sign-in-with-your-atlassian-account Teams already using Jira or Confluence can now access TofuPilot with their existing Atlassian account. - Added Atlassian sign-in on the login and sign-up pages, and to the connected accounts settings - Documented the self-hosted setup with the OAuth app configuration and environment variables Note: requires an Atlassian account with access to a Jira or Confluence site. ### See run outcomes on the measurement chart URL: https://www.tofupilot.com/changelog/see-run-outcomes-on-the-measurement-chart The measurement control chart showed each point's own pass/fail, but seeing how runs behaved meant filtering by run outcome and reloading, losing the full distribution. - Added a colored ring around every control chart point showing its run's outcome (pass, fail, error, timeout, aborted), so value-to-run-outcome correlation is visible in one view - Hid the ring only where it would blend into the point's own color, and unified point sizes so ringed points don't read as larger values ### Fix test runs freezing when realtime is blocked URL: https://www.tofupilot.com/changelog/fix-test-runs-freezing-when-realtime-is-blocked On locked-down factory networks where the realtime streaming server is unreachable (missing DNS record, firewalled WebSockets), test runs and the station service could freeze forever before starting, with no error and no log. - Fixed runs and the station service to start immediately in every case: the dashboard live view now connects in the background and can never block a run, the operator UI, or result uploads. If it cannot connect within 10 seconds, a clear warning explains what to check and the station keeps working locally - Fixed stations where a user login coexisted with the station identity: deployment runs now always use the station identity, restoring the dashboard live view that was silently disabled - Added self-identifying diagnostics: every run prints its CLI version and writes a per-run log from the very start, so support screenshots and log files always show which binary ran and what happened ### Cut drift alert noise to one alert per event URL: https://www.tofupilot.com/changelog/cut-drift-alert-noise-to-one-alert-per-event One production incident — a failing fixture, a drifting environment — used to raise a separate drift alert for every affected measurement, and brief wobbles could alert and resolve within minutes. - Grouped automatic drift alerts into one event per procedure and part: every drifting measurement attaches to the same alert with its own timeline, and same-day recurrences reopen it instead of opening a duplicate. Custom alert rules are unchanged. - Added persistence gates: a drift now has to hold across consecutive runs before it alerts — and before it auto-resolves — ending one-off wobble alerts and resolve/reopen flapping. Sudden critical shifts still fire immediately. - Added an instrument-resolution floor: a shift smaller than the measurement device's own step no longer alarms, silencing last-digit noise on quantized sensors. ### Investigate quality incidents with AI URL: https://www.tofupilot.com/changelog/investigate-quality-incidents-with-ai Every alert can now launch an AI root-cause investigation on demand — testing hypotheses against production data and reporting what actually changed. - Added a root-cause investigation to every alert, started with one click: it builds a live hypothesis tree scoped to the incident window, tests each lead against baseline data, and shows the supporting evidence as charts. - Added a written conclusion that highlights the most likely cause(s), with recommended next steps split into immediate fixes and longer-term improvements grounded in the findings. - Added drift attribution that pinpoints whether a measurement shifted in level, spread, or ranking, plus incident-vs-baseline checks for outcome shifts and retest churn. - Added stop and re-run controls for investigations, with results visible to everyone in the organization. ### Keep date range when switching pages URL: https://www.tofupilot.com/changelog/keep-date-range-when-switching-pages Switching between analytics pages used to reset your selected date range back to the default, forcing you to re-pick it on every page. - Added carry-over of the selected date range when navigating between Runs, Run Analytics, Phase Pareto, Measurement Control and API Logs - Kept default behavior unchanged when no range is selected, so shared links stay relative ### Upload and view attachments behind a firewall URL: https://www.tofupilot.com/changelog/upload-and-view-attachments-behind-a-firewall Factory networks behind strict proxies had to whitelist third-party storage domains to use run attachments; every attachment URL now stays on TofuPilot domains. - Added attachments.tofupilot.app: uploads and downloads from SDKs, stations, and the dashboard no longer touch external storage hosts - Added a 95 MB per-attachment limit with a clear error message, replacing an undocumented unlimited behavior - Improved attachment serving with video seeking support and stricter content-type protections ### Clarify procedure and station setup tabs URL: https://www.tofupilot.com/changelog/clarify-procedure-and-station-setup-tabs The Get Started tabs on procedure and station pages now say what each integration path actually does, so new users can pick between the CLI and the SDKs at a glance. - Renamed the setup tabs to "TofuPilot runs your tests" and "Your code sends results" on both pages, with icons matching each path - Improved the station setup descriptions: supported frameworks and SDK languages are now listed inline in a single readable sentence - Updated the API key empty state to a shorter, clearer call to action ### Fix attachment size showing 0 on the dashboard URL: https://www.tofupilot.com/changelog/fix-attachment-size-showing-0-on-the-dashboard Attachments uploaded through the SDKs showed a size of 0 and no file type on the dashboard, even though the file itself was stored and downloadable. - Fixed the server to record each attachment's size and content type when an SDK links an uploaded file to a run - Fixed the C#, Rust, C++ and MATLAB upload helpers to finalize uploads, and the Python client to declare file size and type up front - Improved upload error reporting: a stored file that fails to link to its run is now reported instead of logged as success ### Add private CA support to live stations URL: https://www.tofupilot.com/changelog/add-private-ca-support-to-live-stations The `--ca-cert` certificate from CLI 1.2.0 only covered HTTP, so stations behind a corporate CA uploaded runs but showed as offline. - Added private CA support to the live connection, so one flag covers every connection. - Added the system certificate store as a trust source, so a CA installed by IT works without the flag. - Fixed the "check DNS" message hiding the real certificate error. ### Slim OpenHTF uploads, trust private CAs URL: https://www.tofupilot.com/changelog/slim-openhtf-uploads-trust-private-cas Stations that only upload no longer install a full test framework, and self-hosted instances behind a corporate certificate authority work end to end. - Removed the `TofuPilot` context manager and its streaming stack from the Python client — uploads go through the `upload()` output callback, and the CLI provides the live operator UI. - Made OpenHTF an optional extra: install `tofupilot[openhtf]` for the callback, plain `tofupilot` for API-only stations. - Added private CA support to the CLI: pass `--ca-cert` at login or set `TOFUPILOT_CA_CERT`, and the saved certificate is reused by unattended stations across logins. - Neutralized leftover `upload()` callbacks under `tofupilot run`, so migrated procedures work unedited without crashing on a missing API key or creating duplicate runs. - Fixed custom CA bundles replacing the trust store instead of extending it, which broke attachment uploads to publicly-signed storage hosts. ### Read full measurement names in pareto charts URL: https://www.tofupilot.com/changelog/read-full-measurement-names-in-pareto-charts Long measurement and phase names were cut off after 24 characters in Measurement Control and Phase Pareto, so bars that differed only by frequency band or test temperature all looked identical and there was no way to tell which one to click. - Added a name column that sizes itself to the longest name on screen, with a drag handle to widen or narrow it and a double-click to return to automatic - Improved shortening to keep both ends of a name when space runs out, so measurements that differ only by their suffix stay distinguishable - Added the execution count and Cpk value at the end of each bar, and kept the value scale pinned in view while scrolling through long lists ### Fix MCP client connection after authorization URL: https://www.tofupilot.com/changelog/fix-mcp-client-connection-after-authorization Connecting an AI client over MCP could stall after clicking Authorize, leaving the client waiting even though access was granted. - Fixed the authorization hand-off so the client receives its connection code reliably on the first try - Removed the confusing "finish connecting manually" step from the authorization screen ### Restore the OpenHTF TofuPilot context manager URL: https://www.tofupilot.com/changelog/restore-the-openhtf-tofupilot-context-manager Bench scripts using `with TofuPilot(test):` work again without code changes, while a deprecation warning points to the recommended upload callback and CLI operator UI. - Added the `TofuPilot` context manager back as a thin wrapper around the `upload` output callback, keeping the historical quiet Ctrl-C exit - Fixed duplicate uploads when reusing one test object across several runs, and restored console quieting between runs - Fixed attachment uploads to always verify TLS with the client's CA bundle on self-hosted instances with a private CA ### Compare multi-dimensional curves across runs URL: https://www.tofupilot.com/changelog/compare-multi-dimensional-curves-across-runs Measurement Control now charts measurements that record a whole curve per run rather than a single value — a temperature ramp, a current draw over time, a sweep across setpoints. - Added an Individuals view that overlays one curve per run, colouring golden and failing reference units automatically and keeping the rest as a grey population. Selecting a curve selects its row in the table below, and vice versa. - Added a Band view that summarises every run in the window as a p05–p95 envelope with its median, so a single unit can be read against the whole fleet at production volume rather than against a wall of overlapping curves. - Added spec limits drawn from the min and max aggregations already attached to a series, and an optional second series on its own right-hand axis. ### Version your procedures from procedure.yaml URL: https://www.tofupilot.com/changelog/version-your-procedures-from-procedureyaml The `version` in your `procedure.yaml` is now the version recorded with every run. It used to be ignored — runs read `pyproject.toml` instead, so monorepo procedures recorded no version at all. - Changed the recorded version to come from `procedure.yaml`. OpenHTF, pytest and Robot Framework procedures keep using `pyproject.toml`. - Made `version` optional. Procedures without one run normally and are not linked to a version. ### Track each curve axis on its own control chart URL: https://www.tofupilot.com/changelog/track-each-curve-axis-on-its-own-control-chart A multi-dimensional measurement records several quantities across one test, and each of them now gets its own control chart in Measurement Control. - Added one control chart per axis, so an average, minimum or maximum is tracked against the quantity it actually belongs to. - Added series names to the chart's axis selectors, so a curve is picked by what it measures rather than by its unit alone. - Extended the second Y axis to the whole population of runs, so a comparison between two quantities is visible before any unit is selected. ### Pass private CA certificates as PEM bytes URL: https://www.tofupilot.com/changelog/pass-private-ca-certificates-as-pem-bytes `add_root_certificate` now takes the PEM bytes and returns a `Result`. It accepts a bundle, so a root and its intermediates can be passed together. - Changed `add_root_certificate` to take PEM bytes; certificates added through it are now applied to the trust store - Added the bundled certificate authorities alongside the system trust store, so a container without a system certificate bundle keeps connecting - Added a connect timeout to attachment transfers ### Add Bitbucket support to all five SDKs URL: https://www.tofupilot.com/changelog/add-bitbucket-support-to-all-five-sdks All five official clients are generated from one V2 specification, so they stay consistent with each other and with the API. - Added Bitbucket alongside GitHub and GitLab wherever a repository provider is returned, so procedures and deployments connected through Bitbucket now read correctly. - Improved aggregation values in the C# client to carry any measured type, instead of text only. - Documented the order phases are returned in: the attempts of a retried phase always arrive in attempt order. - Removed the conflict error from the station create and update paths, since station names are deliberately not unique and the API never returns it. ## News ### TofuPilot Supported by Innosuisse URL: https://www.tofupilot.com/news/tofupilot-supported-by-innosuisse TofuPilot receives coaching support from Innosuisse, Switzerland's innovation agency, to accelerate hardware development. ![TofuPilot Supported by Innosuisse](https://bfusqmdwknceg6ih.public.blob.vercel-storage.com/news/2024-02-22-tofupilot-supported-by-innosuisse-featured-image-D69rRqcL6DnUBUl0IE2dhAndKpkT8H.png) TofuPilot has been accepted into Innosuisse's startup coaching program. Innosuisse is Switzerland's federal innovation agency. Their support means structured coaching, credibility with institutional partners, and a path toward full R&D funding. Concretely, we're using the coaching to sharpen our go-to-market and prepare for their next funding stage. The goal: make hardware development faster and more efficient for every team using TofuPilot. More updates soon. ### Enlightra x TofuPilot: Photonic Testing URL: https://www.tofupilot.com/news/enlightra-x-tofupilot-scaling-photonic-device-testing-from-lab-to-production How Enlightra uses TofuPilot to manage photonic device testing from EPFL lab prototypes to Y Combinator-backed production. ![Enlightra's engineering team using TofuPilot for their development process](https://bfusqmdwknceg6ih.public.blob.vercel-storage.com/news/enlightra-s-engineering-team-is-using-tofupilot-everyday-from-the-lab-to-project-follow-up-meetings-r3b1n7H3uGirorbZGDJ8WcwnAaXPLF.png) Enlightra builds multicolor comb lasers for data center optical interconnects, enabling 100x faster data transfer and 10x less energy. They went from EPFL research lab to Y Combinator-backed production, and they use TofuPilot to manage their test data along the way. ![Enlightra chipscale photonic engine on a test rig.](https://bfusqmdwknceg6ih.public.blob.vercel-storage.com/news/enlightra-chipscale-photonic-engine-on-a-test-rig-Lyp16TDisROsVRLEgdBwuoAvwgp2bE.png) ## The Problem Photonic devices require validation across optical performance, thermal characteristics, and electrical interfaces. Each prototype has dozens of precision components with tight tolerances. Before TofuPilot, the Enlightra team was dealing with: - Test data scattered across multiple instruments and file formats - Manual correlation between design specs and measured performance - Inconsistent protocols making results hard to compare across builds - Limited traceability from results back to component batches ![Johana Bernasconi, Lead Engineer at Enlightra.](https://bfusqmdwknceg6ih.public.blob.vercel-storage.com/news/johana-bernasconi-lead-engineer-at-enlightra-W3pvVRqcfp7OooPu90u7gCYlKwGKvt.png) > We needed a systematic approach to test data management that could scale with our development velocity. TofuPilot provided the infrastructure we needed to maintain engineering rigor while moving fast. > > — **Johana Bernasconi**, Lead Engineer at Enlightra ## What They Built Within two days the team had: - **Prototype traceability**: Each device build linked to component batch records, assembly procedures, and design revisions - **Standardized test protocols**: Consistent validation sequences with automated parameter logging across engineers - **Automated analysis**: Direct correlation between measured performance and specifications with pass/fail determination ![Lou Kanger, Engineer at Enlightra is using TofuPilot to track prototypes of their devices.](https://bfusqmdwknceg6ih.public.blob.vercel-storage.com/news/lou-kanger-engineer-at-enlightra-is-using-tofupilot-to-track-prototypes-of-their-devices-5uWXpuVqLXP7kpAaWrZJeqtBob9Hv7.png) ![The TofuPilot Units feature allows for precise tracking of units and validation results.](https://bfusqmdwknceg6ih.public.blob.vercel-storage.com/news/the-tofupilot-assets-feature-allows-for-precise-tracking-of-assets-and-validation-results-tYV7SLMMgUqILDG5WKbD3MNJvBG21O.png) ## The Result When optical power measurements showed variation across device batches, the team could trace issues to specific manufacturing steps and suppliers. That level of traceability wasn't possible before. > The visibility into our test data has changed how we approach design iterations. We can now make data-driven decisions about design changes and identify optimization opportunities that weren't visible before. > > — **Johana Bernasconi**, Lead Engineer at Enlightra TofuPilot now manages Enlightra's complete validation pipeline: component characterization, packaged device testing, and system-level integration. Test data flows automatically from instruments into structured records, with no manual transcription. If your team is scaling from lab to production, we'd love to hear how we can help. ### OpenHTF Integration, Docs, and Discord URL: https://www.tofupilot.com/news/openhtf-integration-docs-tofupilot-discord TofuPilot adds native OpenHTF support, launches openhtf.org documentation, and opens a Discord community for test engineers. Many of you have asked us how to standardise hardware test execution. We know how time-consuming building your own test framework can be. We've been there. Luckily, some hardware folks at Google had the same problem and built OpenHTF. Today, we're shipping native support for OpenHTF tests in TofuPilot. If you're using your own logic with our Python client, nothing changes. But if you're looking for a test framework or already using OpenHTF, you should give it a try. ## OpenHTF in TofuPilot With just **one line** added to your OpenHTF scripts, our open-source Python client automatically creates a run on TofuPilot, parsing the test phases, measurements, and attachments, while storing the original OpenHTF JSON log for traceability. ![Python code example showing OpenHTF test script with one-line TofuPilot integration for automatic test result upload](https://bfusqmdwknceg6ih.public.blob.vercel-storage.com/news/2024-10-21-openhtf-integration-docs-and-tofupilot-discord-python-script-ty8KZyG7G3jSvZUVZnJaGpA06KrEiE.png) Check out the OpenHTF integration section in our docs for details. ## A Great Framework Deserves Great Docs OpenHTF is robust, built from the ground-up for hardware testing, but it lacked proper documentation. So, we rolled up our sleeves and created openhtf.org, a complete documentation for the framework, packed with screenshots, code snippets and advanced use cases. ![OpenHTF documentation website homepage showing comprehensive guides code examples and framework tutorials at openhtf.org](https://bfusqmdwknceg6ih.public.blob.vercel-storage.com/news/2024-10-21-openhtf-integration-docs-and-tofupilot-discord-openhtf-documentation-89NUxKSCLF8xzkfb74LPK35UaNPKXF.png) Of course, we couldn't resist adding some TofuPilot integration examples. ## Don't Build Tests Alone We're on Discord and would love to have you join us for insights from fellow test engineers and real-time connections with our team. If Discord isn't your thing, no worries, you can always reach us at support@tofupilot.com. ![TofuPilot Discord community invitation banner with join link for hardware test engineers](https://bfusqmdwknceg6ih.public.blob.vercel-storage.com/news/2024-10-21-openhtf-integration-docs-and-tofupilot-discord-discord-community-FKlo8R2RriEiU6zaxRp9jygGAksMTx.png) ### TofuPilot Backed by FIT Innovation Foundation URL: https://www.tofupilot.com/news/tofupilot-backed-by-fit-innovation-foundation TofuPilot receives support from the FIT Innovation Foundation, a Swiss deep-tech program, to accelerate product development. TofuPilot has received support from the FIT Innovation Foundation, a Swiss tech program that backs early-stage deep tech. The funding will go directly into product development. More features, faster. ![TofuPilot and FIT Innovation Foundation logos](https://bfusqmdwknceg6ih.public.blob.vercel-storage.com/news/2024-12-02-pro-plans-self-hosting-and-swiss-innovation-award-fit-foundation-logo-JpAJc3MW6ep8Pj6UQIr1pLJyC1EnUB.png) Thanks to the FIT team for the support. ### Free and Paid Plans, and Self-Hosting URL: https://www.tofupilot.com/news/pro-plans-self-hosting-swiss-innovation-award TofuPilot introduces free and paid plans with a Lab free tier, Pro at $50/month, and Docker-based self-hosting for enterprise teams. Many of you have asked: "What's your business model?" and "Can we self-host TofuPilot?" This week, we're answering both. ## Free & Paid Plans Defining pricing while balancing simplicity, accessibility, and sustainability took time. After gathering your feedback, here's what we landed on: one free tier and two paid plans. The **Lab Plan** is free. It includes all key features and supports small production volumes, for as long as you want. The **Pro Plan**, starting at $50/month, adds secure collaboration, external test deployments, generous run and attachment volumes, and scales with your growth. The **Enterprise Plan** covers self-hosting, annual billing, dedicated support, and custom needs. ![TofuPilot pricing page showing three plan tiers Lab free tier Pro plan at $50 per month and Enterprise plan with custom pricing](https://bfusqmdwknceg6ih.public.blob.vercel-storage.com/news/2024-12-02-pro-plans-self-hosting-and-swiss-innovation-award-pricing-page-PlOXpVctwYdqt2i0oLIQkIQt3LZDJD.png) ### Smooth Transition No action needed. Free accounts will automatically switch to the Lab Plan. Paying organizations will transition based on their current setup. If there are billing changes, we'll notify you directly. ## Self-Hosting All plans include secure cloud hosting by default. But some organizations face connectivity constraints or IT policies that require on-premises deployment. TofuPilot Self-Hosting is now available as part of the Enterprise Plan. ![TofuPilot self-hosting documentation showing Docker deployment instructions and on-premises setup configuration guides](https://bfusqmdwknceg6ih.public.blob.vercel-storage.com/news/2024-12-02-pro-plans-self-hosting-and-swiss-innovation-award-self-hosting-docs-OO0r1slZ2wWlYx5lOHcUb7RMxgvtDJ.png) ### Multi-Measurement Support and IMU Calibration URL: https://www.tofupilot.com/news/multi-measurement-support-and-imu-calibration-template TofuPilot now supports measurements and multi-measurements for OpenHTF and Python client, plus a new IMU calibration template. TofuPilot now supports measurements, one of the most requested features. This means better analytics for OpenHTF users and a simpler way to write test scripts with the Python client. ## Measurements & Multi-Measurement Hardware tests usually follow a common structure: data is collected from the Device Under Test or an instrument, errors are logged, then specific measurements are performed and validated before moving to the next phase. ![TofuPilot measurement analytics dashboard displaying voltage timeseries data with pass fail status and validation ranges](https://bfusqmdwknceg6ih.public.blob.vercel-storage.com/news/2024-12-20-multi-measurement-support-and-imu-calibration-template-measurement-features-vpR1cYjdXbAi63z7eNKuUDnQI7LmrX.png) For example, we acquire a voltage timeseries from a multimeter, compute the average voltage, and verify it falls within the specified range of 3.1–3.5V. Frameworks like OpenHTF maintain this structure by separating: **Phases**: Test steps that perform actions and return a pass, fail, or skip status. **Measurements**: Specific values captured during phases with optional validators. TofuPilot now fully supports this data model, making your test scripts shorter and enabling more granular analytics. Measurements are a major change but support incremental adoption on your side. **OpenHTF users**: No action needed. Tests with multi-measurements now sync automatically. **Python client users**: You can use the new *phases* argument in the *create_run* function. Steps will be deprecated but remain accessible for now. Existing steps have been migrated automatically. ## Template: IMU Thermal Calibration Many of you asked for code samples. Here's our first open-source template: calibrating an IMU for temperature on the production line. With two sensors, three axes each, and 36 measurements across two phases, it shows what multi-measurement can do with OpenHTF and TofuPilot. It's a bit nerdy, but we're sure you'll find it useful. More templates are coming. Share your ideas with us. ![Industrial thermal calibration chamber for IMU sensor testing with precise temperature control system](https://bfusqmdwknceg6ih.public.blob.vercel-storage.com/news/2024-12-20-multi-measurement-support-and-imu-calibration-template-thermal-chamber-duPE3RVygb7JSGmGgxbIDn5F0pXfbC.png) ## TofuPilot's 2024 TofuPilot is now live in 10 factories, processing thousands of runs per month. We won 3 innovation awards, incorporated the company, shipped 30+ releases, and moved into new offices at the Unlimitrust Campus in Switzerland. ![Modern glass entrance of Unlimitrust Campus innovation center in Prilly Switzerland where TofuPilot offices are located](https://bfusqmdwknceg6ih.public.blob.vercel-storage.com/news/2024-12-20-multi-measurement-support-and-imu-calibration-template-unlimitrust-campus-edpOnWiXkZvYvfTUr8LVSMBhp3H34r.jpg) See you in 2025. ### Explore TofuPilot Without Writing Any Code URL: https://www.tofupilot.com/news/explore-tofupilot-without-writing-any-code Explore TofuPilot features instantly with one-click demo scripts — no coding required. Plus updates on OpenHTF maintainers and open positions. TofuPilot is built to be easy to integrate into your test scripts. But for users wanting to evaluate it quickly, setting up an IDE, installing a package, and writing a script was too many steps. ## Try TofuPilot with one click ![TofuPilot's Welcome Aboard page, allowing users to try the platform without writing code.](https://bfusqmdwknceg6ih.public.blob.vercel-storage.com/news/2025-02-18-explore-tofupilot-without-writing-any-code-welcome-aboard-page-WduVqw0M6NPXgxcbCAo9StCq9VVz7L.png) We added buttons in the web app to run sample OpenHTF and vanilla Python test scripts in a virtual console. You can create mock-up test data and explore TofuPilot's features with a single click from the new Welcome Aboard page. The scripts running in the background are visible on the page and can be copied as snippets to kickstart your own development. ## We met OpenHTF maintainers OpenHTF is the most popular open-source hardware testing framework. We used it in our previous lives manufacturing drones and recommend it if you don't have an internal solution. But we weren't sure who maintains it today or what its future holds. Our team flew to Mountain View to meet OpenHTF's founders, John and Joe, who originally built it at Google while working on the Glass project. We also met Akash, who leads its current maintenance at Waymo. A team of five engineers actively maintains the framework, with no internal forks or breaking changes expected. ![OpenHTF founders John and Joe meeting with TofuPilot founders Charlotte and Julien at Google campus in Mountain View](https://bfusqmdwknceg6ih.public.blob.vercel-storage.com/news/2025-02-18-explore-tofupilot-without-writing-any-code-openhtf-founders-photo-24Rh1rvTn4CxqNOVTHEX5CekSEZ5Fo.png) We're looking forward to working more closely with them, helping improve documentation, assisting users, and contributing to the framework's future. ## We're hiring The past few months have stretched the team. We're growing with two open positions: **Full-Stack Software Engineer** and **Data Scientist Intern**. They'll join us in our new offices and help build more features for you. If you know someone looking, send them to our careers page. See you next month. ### TofuPilot at EPFL Startup Champions Seed Night URL: https://www.tofupilot.com/news/selected-for-the-epfl-startup-champions-night TofuPilot pitched at EPFL Startup Champions Seed Night in Lausanne, Switzerland's top deep-tech startup showcase. ![TofuPilot at EPFL Startup Champions Seed Night](https://bfusqmdwknceg6ih.public.blob.vercel-storage.com/news/2024-03-21-selected-for-the-epfl-startup-champions-night-featured-image-gUdPk6pNYP5ikvUGzqsy1pwrVBPN89.png) TofuPilot was selected to pitch at EPFL Startup Champions Seed Night, a showcase for deep-tech startups from the EPFL ecosystem in Lausanne, Switzerland. We presented TofuPilot to investors, founders, and engineers building hardware products. The event brought together 12 startups across robotics, medtech, photonics, and space, all solving real engineering problems. - **Event**: EPFL Startup Champions Seed Night - **Date**: Thursday, 25 April 2024 - **Time**: 17:30 - 21:30 GMT+2 - **Location**: Forum Rolex (Rolex Learning Center EPFL), 1015 Lausanne Thanks to the EPFL Innovation Park team for the selection. ### Histograms, Assembly History, and Batches URL: https://www.tofupilot.com/news/test-histogram-assembly-history-batch-number-support-more TofuPilot adds test histograms, date filters, sub-unit assembly tracking, string control charts, and batch number support for production teams. You asked for more analytics after our August release. Here's what shipped in September. ## Visualize Distribution with Histograms We've added a histogram next to the control chart for numeric steps. This lets you see the distribution of measurements across units and runs, and compare them with test limits and 3-sigma values. No code changes needed. Histograms appear automatically for any numeric step. ![Screenshot of TofuPilot's histogram feature, showing measurement distribution and Gaussian curve.](https://bfusqmdwknceg6ih.public.blob.vercel-storage.com/news/2024-10-03-test-histogram-assembly-history-batch-number-support-and-more-histogram-feature-cohL7FMZExePyq4hqS56wQfGYSSywl.png) ## Date Filters for Time-Based Comparisons You can now compare test results across different time ranges. Available on all analytics pages, no setup required. ![Screenshot of TofuPilot's date filter, allowing selection of test results across different time ranges.](https://bfusqmdwknceg6ih.public.blob.vercel-storage.com/news/2024-10-03-test-histogram-assembly-history-batch-number-support-and-more-date-filter-k6rhN1HSKg6rx8L0mnmrs6qdWkdQbf.png) ## Track Sub-Unit Activity You can now track any assembly or rework timeline with the new units activity panel. Operations are logged automatically from test runs or through new dedicated API endpoints. Using the Python client? Pass `sub_units` in your `create_run` call. The activity panel populates automatically. ![Screenshot of TofuPilot's units activity panel, displaying logged operations from test runs and API inputs.](https://bfusqmdwknceg6ih.public.blob.vercel-storage.com/news/2024-10-03-test-histogram-assembly-history-batch-number-support-and-more-units-activity-panel-JVFTAgB8BRzGdiQ7JetR1nX8UHmbZg.png) ## Control Charts for String Steps You can now log and track text-based measurements, such as firmware versions and component IDs, and visualize their distributions using new control charts for string data. These work the same way as numeric charts. Just pass string values in your measurements. ![Screenshot of TofuPilot's control charts for string data, visualizing distributions of firmware versions.](https://bfusqmdwknceg6ih.public.blob.vercel-storage.com/news/2024-10-03-test-histogram-assembly-history-batch-number-support-and-more-string-control-charts-VrrSgVMTzf5Lb9ojVx2ru40PHiGxQ0.png) ## Batch Number Support You can now add batch numbers alongside unit serial numbers and compare test results between batches. Just pass `batch_number` in your `create_run` call. ![Python code showing TofuPilot API usage for tracking batch numbers with unit serial numbers for production traceability](https://bfusqmdwknceg6ih.public.blob.vercel-storage.com/news/2024-10-03-test-histogram-assembly-history-batch-number-support-and-more-batch-numbers-python-ViiD55FaR0bTipyGPKm2iiauH5Q3C9.png) We'd love to hear your thoughts on these updates. ### TofuPilot Framework is here URL: https://www.tofupilot.com/news/tofupilot-framework-is-here TofuPilot Framework launches as an open-source industrial test orchestrator with TofuPilot Studio desktop IDE for building hardware test procedures. Developing hardware tests shouldn't require vendor lock-in, extensive custom development, or Windows-exclusive deployment options. Today's release introduces two complementary products: ## TofuPilot Framework This open-source industrial test orchestrator enables you to: - Execute Python test phases sequentially or in parallel - Create operator interfaces without frontend development expertise - Integrate hardware resources through reusable plugs - Record measurements and attach files with automatic report generation - Test multiple units concurrently ![Framework Overview](https://bfusqmdwknceg6ih.public.blob.vercel-storage.com/news/newsletter-28c5c5a4-f726-7b93-0fda-92f1d88c632c-mUpfE5WxvU6o1Z3PuRUvW5psraOAeM.png) ## TofuPilot Studio This free lightweight IDE (available for Windows, Linux, and macOS) accelerates test development by providing: - Visual operator UI editing - Test sequence flow configuration - Phase and plug execution/debugging - Report synchronization to TofuPilot Dashboard ![Studio Interface](https://bfusqmdwknceg6ih.public.blob.vercel-storage.com/news/newsletter-27073aba-7265-1358-6d5b-ff82c8a0d961-qaB1Fz89GqzhdY0VqeWGGSiocF1dLo.png) ## Getting Started Download TofuPilot Studio for your operating system, select a template, and begin constructing test procedures immediately. ![Getting Started Guide](https://bfusqmdwknceg6ih.public.blob.vercel-storage.com/news/newsletter-31f3a438-31a7-9d9f-bb2a-301cd59a9548-6dcv2azvFVykJVjNEAQW3gXFUEgKWO.png) ## Community Connect with the community on Discord, explore the source on GitHub, or check the docs. All available from tofupilot.com. Happy holidays, and happy testing. ### 1D, 2D, and ND Measurement Arrays URL: https://www.tofupilot.com/news/launch-week-multi-dimensional-measurements-day-2 TofuPilot Launch Week Day 2: Upload and visualize 1D, 2D, and multi-dimensional measurement arrays with graph previews and CSV export. Yesterday, your test logs found a home. Today, measurements arrays get the same attention. ## Tuesday: Multi-dimensional Measurements Until now, measurements in TofuPilot were limited to single values. But real-world tests often produce more complex data: sampled over time, across multiple axes, or as full waveforms. TofuPilot now supports 1D, 2D, and ND measurement arrays. ![TofuPilot multi-dimensional measurements interface with graph previews](https://bfusqmdwknceg6ih.public.blob.vercel-storage.com/news/tofupilot-multidim-screenshot-yZgkn5EtQXCQm0x5LIOqYc59ENWpLh.png) You can explore multi-dimensional measurements directly in the run interface. TofuPilot enables: - Full graph previews of time-series and array data - Toggle visibility for individual measurements - Export data series as CSV files for external analysis Using OpenHTF? You don't need to change anything. OpenHTF native multi-dimensional measurements are now processed for all new runs. Using the vanilla Python client? Just pass a list or array in your measurements. Our updated documentation includes a simple snippet to get you started. Multi-dimensional measurements are automatically included in analytics for duration and yield, but Cpk isn't available yet as it requires custom validators - support for that is coming soon. ## Inside TofuPilot: Backed by Switzerland's Startup Ecosystem TofuPilot wouldn't exist without the support of Switzerland's top innovation programs. Venture Kick and La FIT provided our first funding to get started. EPFL, Innosuisse, and the Canton of Vaud connected us with expert coaching, financial support, and a strong network of tech founders. ![Summary of TofuPilot's sponsors and supporters in Switzerland's startup ecosystem](https://bfusqmdwknceg6ih.public.blob.vercel-storage.com/news/tofupilot-sponsors-summary-RfwZ6w1PJWAgWPlp77KUFIB3cMRyBI.png) Thanks to all these programs for their early support. See you tomorrow! ### Operator UI for the Factory Floor URL: https://www.tofupilot.com/news/launch-week-operator-ui-day-3 TofuPilot Launch Week Day 3: A plug-and-play Operator UI with real-time test streaming and browser-based operator input for the factory floor. Console logs and prompts work fine for developers, but end users of test scripts, operators and technicians, need clearer interfaces. Building custom UIs from scratch is costly and time-consuming. ## Wednesday: Operator UI The new TofuPilot Operator UI provides a plug-and-play web interface that eliminates the need for custom UI development. Key features: - Real-time streaming of test phases, measurements, logs, and metadata - Browser-based user input through OpenHTF's native plug - Secure, permission-controlled real-time connections ![Operator UI Interface](https://bfusqmdwknceg6ih.public.blob.vercel-storage.com/news/newsletter-ed2630f1-4ca1-5eae-23be-bbfb4275dc13-7IsFI6W1vqYd3RXOvE7HbLadrtRf1Q.png) The Operator UI works with any OpenHTF script. Just update your tofupilot package, run your test, and real-time updates appear automatically. Prompts work through the existing `user_input` plug without additional integration code. Scripts using the vanilla Python client aren't yet supported but are planned for a future release. ## Looking Ahead: Role-Based Permissions The Operator UI fully integrates with TofuPilot Stations for real-time monitoring. Upcoming features include: - Granular permissions by user role - Viewer-only accounts with new pricing - External deployment capabilities with maintained control ![Feature Preview](https://bfusqmdwknceg6ih.public.blob.vercel-storage.com/news/newsletter-a8d22b3c-a4dc-6c58-b0f7-e2eea8b2fa66-Z75vjtXKoIZEM9Ryreg9Jg8sqnEuUY.jpeg) See you tomorrow for Day 4! ### API Activity Page and Launch Week Recap URL: https://www.tofupilot.com/news/launch-week-api-activity-day-5 TofuPilot Launch Week finale: new API Activity page for monitoring usage, errors, and performance, plus a recap of all week's releases. It's the last day of Launch Week, and we're closing with something every test engineer needs in production: visibility and control over the TofuPilot API. ## Friday: API Activity We know your test stations run in manufacturing environments where updates aren't always easy, and reliability is critical. As the platform grows with new features and performance improvements, we want to help you stay in control. That's why we added the new API Activity page for all users. It lets you: - See who's calling the API, when, and with which client version - Catch errors quickly, with direct links to the related test runs - Stay informed about upcoming changes or deprecations - Track the performance of your API calls over time ![TofuPilot API Activity page showing API usage, errors, and performance metrics](https://bfusqmdwknceg6ih.public.blob.vercel-storage.com/news/tofupilot-api-activity-screenshot-s9G6Jh60kNVdaR8IygXQFPWxSoamxB.png) No code changes needed. The Activity page is available now in your dashboard. We'll always do our best to avoid breaking backward compatibility. If we ever have to, you'll get several weeks' notice to prepare. ## Launch Week Recap Here's everything we shipped this week: - Support for full test log ingestion - Multi-dimensional measurements - A new plug-and-play Operator UI - Our new Pro Plan to support teams in production We added a lot, but also made the ingestion under the hood up to 60% faster. Until next time. Happy testing. ### REST API and Python SDK for Hardware Testing URL: https://www.tofupilot.com/news/test-hardware-faster-with-new-python-sdk TofuPilot launches its REST API, Python SDK, redesigned Testing Hub UI, and improved onboarding to help hardware teams test faster. Uploading test results used to require multiple UI actions. Now it's a single API call. ## API & Python SDK We've built a REST API and a dedicated Python SDK (available on pip) to automate test result uploads from your scripts. With one call you can: - Create a new test run from any procedure - Link runs to units under test, creating them if they're new - Save test run duration and status - Generate a full test report programmatically with our new "Smart Fields" feature ![Python code example demonstrating TofuPilotClient SDK usage for creating units uploading test results and attachments via REST API](https://bfusqmdwknceg6ih.public.blob.vercel-storage.com/news/2024-06-18-test-hardware-faster-with-new-python-sdk-python-client-M3AJYDhYrQzlwHiSTmaF6uIE4khEZp.png) ## New Dashboard With the API enabling significantly more test results, our old UI wasn't cutting it for navigating that volume. We designed a new dashboard from the ground up. ![Screenshot of TofuPilot's new Procedures page displaying weekly runs, tested components, FPY, and average test time per procedure.](https://bfusqmdwknceg6ih.public.blob.vercel-storage.com/news/2024-06-18-test-hardware-faster-with-new-python-sdk-procedures-page-qUOovGGRBszQWT131Dh0c1L6T9on18.png) ## Get Started We've improved onboarding and invite flows, with a new "Get Started" tab accessible from the sidebar. ![TofuPilot Get Started tab in sidebar navigation for quick onboarding and setup guides](https://bfusqmdwknceg6ih.public.blob.vercel-storage.com/news/2024-06-18-test-hardware-faster-with-new-python-sdk-get-started-FE6vefWYsEqMLqWJnOZhQhmihKDalk.png) We'd love to hear your feedback. ### Test Logs in the Run Page URL: https://www.tofupilot.com/news/launch-week-logger-day-1 TofuPilot Launch Week Day 1: View, filter, and search test logs directly in the Run page. Works with OpenHTF logger out of the box. TofuPilot turns one this week. To mark it, we're shipping one feature per day, all from your feedback. ## Monday: Logger View When you run a test, phases and measurements tell part of the story - but console logs provide essential context, helping you trace execution, catch errors, and debug issues effectively. ![TofuPilot Logger UI screenshot showing log filtering and search capabilities](https://bfusqmdwknceg6ih.public.blob.vercel-storage.com/news/tofupilot-logger-ui-screenshot-eEoI2JuWwHUZS6oOxmeqk5AGA8y7KV.png) You can now view logs directly on the Run page in TofuPilot. Easily: - Filter by log level (debug, info, warning, error, critical) - Search by keyword - Sort by timestamp - See source file names and line numbers Using OpenHTF? You don't need to change anything, logs from the OpenHTF logger are now handled automatically. Using the vanilla Python client? Just add a small snippet to your scripts. Our new Logger documentation page will show you exactly how. ## Inside TofuPilot: Doubling the Team in April With more users and growing feature requests, we've expanded our team to move faster. Dehlya and Quentin have joined as full-stack engineers, and Manon as a data engineer focused on ingestion and analytics, all now working from our new offices in Prilly, Switzerland. ![TofuPilot team at their new offices in Prilly, Switzerland](https://bfusqmdwknceg6ih.public.blob.vercel-storage.com/news/tofupilot-team-picture-XhqpDbfsnqKpVz4fe0xcS0qiTW0KLP.png) See you tomorrow with something new! ### API Docs, Sub-Units, and Attachments URL: https://www.tofupilot.com/news/new-api-documentation-sub-units-attachments-support TofuPilot adds sub-unit tracking for assemblies, file attachments for test runs, and launches new developer documentation for its REST API. TofuPilot's API and Python client now support sub-units, attachments, and we've launched new documentation. ## Sub-Units The API now accepts optional sub-unit parameters on the test run endpoint. Assembly test stations can automatically assign sub-units as children of the assembly units being tested, giving you full unit traceability across the assembly line. Sub-units appear in the Units table and on dedicated Unit pages. ![Screenshot of TofuPilot's Unit page showing units, sub-units, serial numbers, and test counts in a tree view.](https://bfusqmdwknceg6ih.public.blob.vercel-storage.com/news/2024-07-02-new-api-documentation-sub-units-and-attachments-support-unit-page-tree-view-or2UVp9T0M0XRCeUgIQx57SqBm8jfc.png) ## Attachments Test scripts often produce files beyond text and numbers: calibration images, graphs, videos. You can now upload them as attachments directly from the Python client. Attachments can be viewed, downloaded, or deleted from the dedicated run page. ![TofuPilot test run page showing file attachments panel with uploaded calibration images graphs and JSON data files](https://bfusqmdwknceg6ih.public.blob.vercel-storage.com/news/2024-07-02-new-api-documentation-sub-units-and-attachments-support-run-page-attachments-kL4V1l4xBbjravCqHmIlecq3gwhl1F.png) ## Documentation Our new documentation is live on tofupilot.com, continuously updated as the platform evolves. ![TofuPilot documentation homepage featuring Getting Started guide with API examples and integration tutorials](https://bfusqmdwknceg6ih.public.blob.vercel-storage.com/news/2024-07-02-new-api-documentation-sub-units-and-attachments-support-documentation-getting-started-rpdPMEcMGYnihexk4P46Qxu67wnPVk.png) We'd love to hear how you're using these. Let us know. ### Real-Time or Offline Test Sync? URL: https://www.tofupilot.com/news/real-time-or-offline-test-sync TofuPilot adds real-time OpenHTF streaming and improved offline upload so hardware teams can sync test data from any environment. When developing or running tests in connected environments, you want to see results immediately. But testing doesn't always happen online and offline uploads need to be possible, too. Our new release improves both scenarios. ## Online Streaming In connected environments, you want test results instantly, not after they finish. Our new release introduces real-time streaming for OpenHTF test scripts, powered by a new secure real-time infrastructure. Tests appear on your TofuPilot Runs page as soon as they start, with real-time step status tracking. ![TofuPilot dashboard showing real-time OpenHTF test execution with live status updates for each test phase](https://bfusqmdwknceg6ih.public.blob.vercel-storage.com/news/2024-11-07-real-time-or-offline-test-sync-openhtf-real-time-NDC5oqXLahxYi0WWXnwX4JbSt7BHQI.png) No code changes are needed on your side. Just update the TofuPilot Python client, and live streaming kicks in automatically. ## Offline Sync TofuPilot now offers improved support for offline uploads when test stations aren't directly connected, allowing offline uploads without losing accuracy. With the optional "started_at" field, analytics reflect the actual test date instead of the upload date, and upload capacity is now up to 1,000 runs per minute. Check out the offline upload section in our docs for more on this use case. ## Updated Docs We've revamped the docs with insights from our work on openhtf.org. You'll find new guides, feature and code examples to help you get the most out of TofuPilot. ![TofuPilot documentation quickstart section featuring step-by-step guides code examples and integration tutorials](https://bfusqmdwknceg6ih.public.blob.vercel-storage.com/news/2024-11-07-real-time-or-offline-test-sync-documentation-quickstart-xWP3BC4TMP5aKNb6PNWD9iUh9wrvQY.png) We'd love to hear how these work for your setup. ### Introducing TofuPilot: Hardware Test Analytics URL: https://www.tofupilot.com/news/introducing-tofupilot-accelerating-hardware-innovation TofuPilot launches as a plug-and-play analytics platform for hardware test data, with OpenHTF and Python SDK support out of the box. ![TofuPilot - Logo Reveal](https://bfusqmdwknceg6ih.public.blob.vercel-storage.com/news/2024-01-28-introducing-tofupilot-accelerating-hardware-innovation-featured-image-uqBTEAUCMxQmNaPS0TTDoXIWjaCOap.png) Over the last few months, we've met over a hundred hardware teams working on complex multi-disciplinary products like autonomous robots, spacecraft, and medical devices. We've heard their feedback: traditional industrial solutions don't match their speed, lack flexibility, and don't fit modern development workflows. On the other hand, generic collaboration tools create unstructured data that's hard to manage, analyse, and connect, slowing down progress as projects grow. This inspired us to build TofuPilot, a plug-and-play analytics platform for hardware test data. You connect it to your test scripts (OpenHTF, pytest, or our Python SDK), and it gives you runs, measurements, FPY tracking, and Cpk analysis out of the box. We picked a name that reflects a departure from traditional tooling: something short, memorable, and distinctive. Our new name comes with a mascot, for which the community is already voting on a name. Welcome aboard TofuPilot. ### TofuPilot Selected by Venture Kick URL: https://www.tofupilot.com/news/tofupilot-selected-by-venture-kick Learn how TofuPilot was selected by Venture Kick, Switzerland's leading startup program, and what it means for product development. ![TofuPilot Selected by Venture Kick](https://bfusqmdwknceg6ih.public.blob.vercel-storage.com/news/2024-01-31-tofupilot-selected-by-venture-kick-featured-image-Pg5CIMMudsf0mjbUDwxHTKzq8dfbXP.png) TofuPilot has been selected by Venture Kick, Switzerland's leading startup funding program. The program provides CHF 150K in equity-free funding across three stages, along with mentoring and access to investor networks. For us, this means more resources to ship features faster and grow the team. For you, it means a more stable product with a clearer path forward. Thank you to everyone who's been part of the journey so far. We'll keep building. ### Rust SDK for Hardware Test Data URL: https://www.tofupilot.com/news/rust-sdk-for-hardware-test-data TofuPilot ships an official Rust client covering the full V2 API with async builders, typed errors, retries, and file upload helpers. ![Rust SDK for TofuPilot hardware test data](https://bfusqmdwknceg6ih.public.blob.vercel-storage.com/news-1775071248267-HxAN3Cf41kVAiOo1Y4H46mOihWvcPl.png) Not every test station runs Python. Some teams need bare-metal performance, memory safety, or just prefer `cargo test` over `openhtf` or `pytest`. TofuPilot now has an official Rust client. ## Create a run The SDK uses async builders. Required fields are enforced at compile time, optional ones chain before `.send()`. ```rust use tofupilot::TofuPilot; use tofupilot::types::*; let client = TofuPilot::new(std::env::var("TOFUPILOT_API_KEY").unwrap()); let run = client.runs().create() .procedure_id("FVT-001") .serial_number("SN-042") .part_number("PCB-V2") .outcome(Outcome::Pass) .started_at(chrono::Utc::now() - chrono::TimeDelta::minutes(5)) .ended_at(chrono::Utc::now()) .send() .await?; ``` ## Phases and measurements You can push structured test data in one call. Builders keep the nested types readable. ```rust let run = client.runs().create() .procedure_id("FVT-001") .serial_number("SN-042") .part_number("PCB-V2") .outcome(Outcome::Pass) .started_at(now - chrono::TimeDelta::minutes(5)) .ended_at(now) .phases(vec![RunCreatePhases::builder() .name("voltage_check") .outcome(PhasesOutcome::Pass) .started_at(now - chrono::TimeDelta::minutes(5)) .ended_at(now - chrono::TimeDelta::minutes(3)) .measurements(vec![RunCreateMeasurements::builder() .name("output_voltage") .outcome(ValidatorsOutcome::Pass) .measured_value(serde_json::json!(3.3)) .units("V") .build() .unwrap() ]) .build() .unwrap() ]) .send() .await?; ``` ## Upload files in one call Attach files directly to runs or units through the `attachments()` sub-resource. ```rust // Upload a file to a run let id = client.runs().attachments().upload(&run.id, "report.pdf").await?; // Upload a file to a unit let id = client.units().attachments().upload("SN-0001", "calibration.pdf").await?; // Download an attachment client.runs().attachments().download(&url, "local-report.pdf").await?; // Delete a unit attachment client.units().attachments().delete("SN-0001", vec![id]).await?; ``` ## Typed errors Every API error maps to a Rust enum. No string matching. ```rust use tofupilot::Error; match client.runs().get().id("nonexistent").send().await { Ok(run) => println!("Found: {}", run.id), Err(Error::NotFound(e)) => println!("Not found: {}", e.message), Err(Error::Unauthorized(e)) => println!("Bad key: {}", e.message), Err(e) => println!("Other: {e}"), } ``` ## Retries built in The client retries on 429 and 5xx with exponential backoff and jitter. You can tune it or leave the defaults. ## Every V2 endpoint Runs, units, parts, revisions, procedures, batches, stations, versions, users. 132 integration tests run on every deployment. The SDK is open source under MIT. Source on [GitHub](https://github.com/tofupilot/rust), package on [crates.io](https://crates.io/crates/tofupilot). ```bash cargo add tofupilot ``` ### Workflows: Tests to Odoo, Linear & More URL: https://www.tofupilot.com/news/workflow-automation-for-test-data TofuPilot now has a visual workflow engine. Trigger actions when test runs complete, route data to Odoo or Discord, and build logic with filters and branches. ![TofuPilot workflow editor showing a visual automation flow triggered by a test run](https://bfusqmdwknceg6ih.public.blob.vercel-storage.com/news/news-workflows-N0V9X18PQdDe1AXj9em8kBrRpmPxeL.png) You can now build workflows that react to your test data. A run comes in, TofuPilot checks conditions, and fires actions: emails, Discord messages, Odoo records, Linear tickets, or any HTTP endpoint. All configured from a visual editor with a settings sidebar for each node. ## Triggers ![Trigger selection panel](https://bfusqmdwknceg6ih.public.blob.vercel-storage.com/news/news-workflows-triggers-RYxA3CBa21FBdIfKcVo0h39zTXbqpF.png) You can start a workflow from any of these events: - **Run created**, **Unit created**, **Part created**, **Revision created**, **Batch created** - **Recurring schedule** with frequency (daily, weekly, monthly), time, and timezone picker - **Manual trigger** ## Actions ![Action palette by category](https://bfusqmdwknceg6ih.public.blob.vercel-storage.com/news/news-workflows-actions-02yAiGpbhEJBtTBBdIYUVEqxc7kWob.png) You can send notifications, push data to external tools, or run utilities. Each action has its own settings panel in the sidebar. **Email**: Pick recipients from your team (with avatar search), write a subject and body. All text fields have a variable explorer that lets you browse and insert fields from the trigger and upstream nodes (serial number, outcome, procedure, part number...). **Discord**: Paste a webhook URL, compose a message with the same variable explorer. **Odoo**: Select a connection, pick a model from a live dropdown (fetched from your Odoo instance), and map fields with a dynamic builder. Three actions available: Create Record, Update Record, Post Message. **Linear**: Select a connection, pick team, project, state, priority, labels, and assignee from live dropdowns. Create Issue or Add Comment. **HTTP Request**: Choose method (GET, POST, PUT, PATCH, DELETE), enter a URL, and write a body with variables. **Utilities**: Delay (configurable duration), Formula (math on variable values), Celebrate (confetti on completion). ## Conditions ![If/Else condition builder](https://bfusqmdwknceg6ih.public.blob.vercel-storage.com/news/news-workflows-filters-m25Vhcjn3FBQMYyho4wU4sQOlOj5Ql.png) You can route test data down different paths based on any field. The condition builder lets you pick a variable, choose an operator, and set a value. - **Filter**: stops the flow unless conditions are met. Supports multiple rules with AND/OR logic. - **If / Else**: splits into two paths based on a condition. - **Switch**: fans out into multiple named branches based on a field value. Each branch gets its own downstream actions. Operators include equals, not equals, contains, starts with, greater than, less than, is empty, and relative date comparisons. ## Example: quality alert on failed run 1. **Trigger**: Run created 2. **Filter**: outcome equals FAIL 3. **Switch** on procedure: - `FVT-001`: email quality team + Linear issue - `ICT-001`: Discord alert to electronics channel - Default: email to test engineering lead Failed tests go to the right people with full context. No glue scripts. ## Example: sync serial numbers to Odoo 1. **Trigger**: Run created 2. **Filter**: outcome equals PASS 3. **Odoo Update Record**: write the serial number to the corresponding production lot in Odoo using the part number to find the record Your serial numbers stay in sync between TofuPilot and Odoo without manual entry. ## Execution tracking ![Execution view with step status and timing](https://bfusqmdwknceg6ih.public.blob.vercel-storage.com/news/news-workflows-executions-Z7PetWsf0lxWlaO9NzWkFMlOnItZkn.png) You can inspect every workflow run step by step. Each execution shows status, duration, errors, and the data that flowed through each node. The view follows the same layout as the editor. Running executions refresh live. Failed steps show the error inline. Filter by status and drill into any execution. ## The editor You can build everything from a visual canvas. Add nodes from the palette, wire them up, and configure each one in the sidebar. Drag to pan, scroll to zoom. Auto-saves. ## What's next InvenTree and Slack integrations are in progress. Webhook triggers (so external systems can start workflows) are next. If you need a specific integration, tell us on Discord. ### Test Steps and Open-Source Python Client URL: https://www.tofupilot.com/news/test-steps-oss-python-client TofuPilot introduces structured test Steps for pass/fail and numeric measurements, plus open-sources the Python client under MIT license. Two updates shipping today: structured test steps and our open-source Python client. ## Introducing Steps Steps allow you to log structured test results, from simple pass/fail checks to advanced numeric tests with limits. ![Steps visualization](https://bfusqmdwknceg6ih.public.blob.vercel-storage.com/news/newsletter-e5cb85c8-697d-f865-ad6d-7eccac807647-ZSRhx8OsR84kNPoBA5hjWZp0DW5Iy5.png) The Run page now displays step-by-step execution details, while the Procedure page aggregates insights across all runs, including average durations and pass rates, with more analytics coming soon. ![Analytics dashboard](https://bfusqmdwknceg6ih.public.blob.vercel-storage.com/news/newsletter-b5f7623e-93fa-887c-d5be-cc43a96acc65-iPYqqCFpwK4ZQUwCtv6PZzNRA1iJ6p.png) You can incrementally adopt Steps in existing scripts without disrupting current workflows. ## Open Source Python Client Our Python client is now open source under the MIT license. Review the implementation before production deployment and contribute through GitHub. The repository is available on GitHub for community contributions and feature requests. ### Test Step Insights and First Pass Yield URL: https://www.tofupilot.com/news/test-steps-insights-first-pass-yield-limits-accuracy-and-more TofuPilot launches Test Step Insights with first-pass yield tracking, limits accuracy analysis, and test duration breakdown. Building in-house test analytics is resource-intensive. TofuPilot's new **Test Step Insights** module gives you manufacturing test analytics out of the box: plug-and-play, no dashboard development needed. The module helps you identify real issues and separate them from false failures caused by test errors. Here's what it covers. ## First-Pass Yield at Step Level First-pass yield is the key metric for test system quality. TofuPilot now provides it in real-time, broken down by step, so you can spot which steps cause the most failures. ![TofuPilot first pass yield chart showing FPY percentage trends over time with step-by-step breakdown for failure analysis](https://bfusqmdwknceg6ih.public.blob.vercel-storage.com/news/2024-08-28-test-steps-insights-first-pass-yield-limits-accuracy-and-more-fpy-over-time-CZfUqgg4wBx3Hd9hyNlD7hMX82sGZJ.png) The module also ranks steps by priority so you know where to focus first. ![TofuPilot test steps analysis with color-coded prioritization showing critical failing steps in red and improvement opportunities](https://bfusqmdwknceg6ih.public.blob.vercel-storage.com/news/2024-08-28-test-steps-insights-first-pass-yield-limits-accuracy-and-more-test-steps-prioritization-7CmI2kt2JidgNUiB7owNASb5x6bm4h.png) ## Limits Accuracy & Deviations Many tests require comparing measured values to precise limits. TofuPilot now automatically computes how well each step meets its limits, giving you a clear view of your procedure's process capability. ![TofuPilot process capability analysis showing Cpk scores with color indicators for test limit accuracy and deviation detection](https://bfusqmdwknceg6ih.public.blob.vercel-storage.com/news/2024-08-28-test-steps-insights-first-pass-yield-limits-accuracy-and-more-process-capability-1Gf3J2mAq8h8XNVezMMV5KwZKqd8Op.png) This makes it straightforward to spot incorrectly set limits or sudden deviations. ## Test Duration Analysis A breakdown of each step's duration helps you identify the time-consuming steps slowing down your test benches. ![TofuPilot test duration breakdown showing execution time analysis for each test step to identify bottlenecks and optimization opportunities](https://bfusqmdwknceg6ih.public.blob.vercel-storage.com/news/2024-08-28-test-steps-insights-first-pass-yield-limits-accuracy-and-more-test-duration-analysis-b4AmsoygOkuGD2qOfxYpPZ0myhTAF3.png) No code changes needed. These analytics appear automatically for any procedure with numeric steps. We'd love to hear your thoughts. ### Stations, Procedure Versions, and Roadmap URL: https://www.tofupilot.com/news/stations-procedure-versions-public-roadmap TofuPilot introduces Stations for secure production deployment, procedure version tracking, and a public development roadmap. Three major features ship today: secure production deployment stations, procedure version tracking, and a public roadmap. ## Stations for Production Deployment ![Stations feature](https://bfusqmdwknceg6ih.public.blob.vercel-storage.com/news/newsletter-93d04979-81cb-1bc2-8183-4d0db282d315-NNfp9EtleRsTdxnsitnVzclGzG0dnv.png) Stations let you deploy test procedures to production securely. Each Station can be associated with one or multiple procedures, providing isolation and security through unique API access with limited permissions. Existing API keys continue to work. Stations add a new, scoped option alongside them. ## Procedure Version Tracking ![Version tracking interface](https://bfusqmdwknceg6ih.public.blob.vercel-storage.com/news/newsletter-be558a0c-6bcc-0d86-f277-b3181e940179-d4jNyheoQ3EsecknztayrNqU1DD9UM.png) The platform now supports logging and filtering by procedure version, making it simpler to pinpoint issues and determine when they first appeared. Using the Python client? Pass `procedure_version` in your `create_run` call. Using OpenHTF? The version is picked up automatically from your script metadata. ## Public Roadmap ![Roadmap announcement](https://bfusqmdwknceg6ih.public.blob.vercel-storage.com/news/newsletter-2699778d-5da3-631c-f550-a151c222a9db-eVsh3ioSWz5fDuiwmO388wL6OOJ7Bv.jpeg) We've published our development roadmap on tofupilot.com. Your feedback shapes what we build next. ### Pro Plan: Stations, Orgs, and Billing URL: https://www.tofupilot.com/news/launch-week-pro-plan-day-4 TofuPilot Launch Week Day 4: The new Pro Plan adds Stations for secure deployment, multi-user orgs, and self-serve subscription management. Today, we're making it easier to scale your testing to production and shape what's next for TofuPilot. ## Thursday: Upgrade to Pro Or stick with Lab - it's free forever. But if you're ready for production, Pro unlocks everything you need. Here's what each plan includes: **Lab:** - 100 test runs /month - 10 GB /month attachment storage - Access to the Analytics and the new Operator UI ![TofuPilot Operator UI web interface screenshot](https://bfusqmdwknceg6ih.public.blob.vercel-storage.com/news/tofupilot-operator-ui-screenshot-EggH1Oxtrhc1paJJaTrGxXLMn6Hfoj.png) **Pro unlocks:** - Stations for secure deployment (scoped API keys tied to procedures) - Multi-user organization support - 1,000 runs /month and 100 GB /month attachments before usage-based billing You can check full plan details on our pricing page. **What's ahead:** - Starting May 16, Pro features will require an active subscription - Existing Pro subscribers will be upgraded automatically We know engineers love nothing more than a good sales call - but with the new Subscription page, you can check your usage and upgrade without one. ![TofuPilot subscription UI showing usage and upgrade options](https://bfusqmdwknceg6ih.public.blob.vercel-storage.com/news/tofupilot-subscription-ui-screenshot-Fndb6TraX8TCRlaMNoJVhNguUwGZxq.png) That said, we'd still love to meet. Our calendar (or inbox) is always open to hear about your challenges, assist you with on-boarding, or chat about what's next. ![TofuPilot mascot at airport gate, metaphor for user signup journey](https://bfusqmdwknceg6ih.public.blob.vercel-storage.com/news/tofupilot-mascot-airport-signup-QvzMyyjVxsCuW4BezeF3QxJPQ2Oych.jpeg) See you tomorrow for the final day of our first anniversary Launch Week! ### TofuPilot now speaks C# URL: https://www.tofupilot.com/news/csharp-sdk-dotnet-hardware-testing TofuPilot now has an official C# client built on community work by @Hylaean, covering the full V2 API with typed responses and errors. ![TofuPilot C# SDK code example showing how to create a test run with measurements from .NET](https://bfusqmdwknceg6ih.public.blob.vercel-storage.com/news/news-csharp-yQynqzRgcZX1u2qrQ5KuDWzgHb3v8h.png) Python isn't the only language on the factory floor. Many hardware teams run .NET for TestStand sequences, LabVIEW integrations, or custom test executors. Until now, they had to wrap raw HTTP calls to use TofuPilot. Our community member **@Hylaean** changed that. He built the **first C# client from scratch**, covering core endpoints with clean async patterns. It was great work, and teams started using it in production. We took it one step further. Starting from his foundation, we auto-generated **full V2 coverage** from our OpenAPI spec, added typed models for every request and response, and wired up **131 xUnit tests** that run on each deployment. The SDK is open source under [MIT license on GitHub](https://github.com/tofupilot/csharp). ## Install You can install the SDK from NuGet: ```bash dotnet add package TofuPilot ``` ## Create a run with measurements You can create a run with phases, measurements, and limits in one call: ```csharp using TofuPilot; using TofuPilot.Models.Requests; var client = new TofuPilot(apiKey: Environment.GetEnvironmentVariable("TOFUPILOT_API_KEY")!); var run = await client.Runs.CreateAsync(new RunCreateRequest { ProcedureId = "FVT-001", SerialNumber = "SN-042", PartNumber = "PCB-V2", Outcome = RunCreateOutcome.Pass, StartedAt = DateTime.UtcNow.AddMinutes(-5), EndedAt = DateTime.UtcNow, Phases = new List { new() { Name = "Voltage Test", Outcome = RunCreatePhasesOutcome.Pass, StartedAt = DateTime.UtcNow.AddMinutes(-5), EndedAt = DateTime.UtcNow, Measurements = new List { new() { Name = "Output Voltage", Outcome = RunCreateMeasurementsOutcome.Pass, MeasuredValue = 3.3, Validators = new List { new() { Operator = ">=", ExpectedValue = RunCreateExpectedValue.CreateNumber(3.0) }, new() { Operator = "<=", ExpectedValue = RunCreateExpectedValue.CreateNumber(3.6) }, }, }, }, }, }, }); ``` ## Upload files in one call Attach files directly to runs or units through the `Attachments()` sub-resource: ```csharp // Upload a file to a run await client.Runs.Attachments().UploadAsync(run.Id, "report.pdf"); // Upload a file to a unit await client.Units.Attachments().UploadAsync("SN-0001", "calibration.pdf"); // Download an attachment await client.Runs.Attachments().DownloadAsync(downloadUrl, "local-copy.pdf"); // Delete a unit attachment await client.Units.Attachments().DeleteAsync("SN-0001", new List { attachmentId }); ``` ## Every V2 endpoint, fully typed Runs, units, parts, revisions, procedures, batches, stations, users. Every resource has full CRUD with typed requests and responses. ## Updated docs with C# examples All [dashboard documentation](https://tofupilot.com/docs/dashboard) pages and every [API reference](https://tofupilot.com/docs/dashboard/api/v2) endpoint now include C# code examples alongside Python. Both are generated from the OpenAPI spec, so they stay in sync as we ship new features. ## What's next We're working on a **NI TestStand integration** so .NET teams running TestStand sequences can push results to TofuPilot without custom scripts. If you're using TestStand or LabVIEW, we'd like to hear how you export test data today. The SDK targets **net10.0 LTS** and is available on [NuGet](https://www.nuget.org/packages/TofuPilot). ### MCP: Connect TofuPilot to Claude, Cursor URL: https://www.tofupilot.com/news/connect-ai-agents-to-your-test-data TofuPilot now has an MCP server. Connect Claude, Cursor, or any MCP client to your test data, authorized in your browser with no API key. ![TofuPilot MCP server connecting Claude, Cursor, and other AI clients to test data](https://bfusqmdwknceg6ih.public.blob.vercel-storage.com/news-1781251536631-W1NREKbKfWZ8mlilreVNnAUiqoRMVR.png) TofuPilot now has an MCP server. Connect Claude, Cursor, or any Model Context Protocol client to your test data and it can read your runs, phases, measurements, logs, units, parts, batches, procedures, stations, deployments, and organization members. With write access, agents can also create, update, and delete runs, units, parts, batches, procedures, and stations, including revisions and versions. No API key to copy or rotate. You authorize once in your browser. ## Connect Your organization settings have a dedicated MCP page with copy-ready snippets for each client. Grab the one you need, authorize in the browser that opens, pick an organization, approve. A token is issued to the client. That's it. **Claude**: `claude mcp add --transport http tofupilot https://www.tofupilot.app/mcp`, then run `/mcp` to authorize. **Cursor**: add the URL to `.cursor/mcp.json`. It opens the browser on first use. **Any other client**: use the endpoint with Streamable HTTP. The client discovers the OAuth flow from the URL. ## Read Every connection can read: runs, phases, measurements, logs, units, parts, batches, procedures, stations, deployments, and organization members. Same filters the API exposes. Ask for the failing units on `FVT-001` this week, the out-of-limit measurements in a run, or the log lines around a failed phase, and the data comes back. ## Write Write is opt-in. When you connect a client, you choose read-only or read and write. With write, the connection can create and update runs, units, parts, batches, procedures, and stations (including revisions and versions). Deletes are there too, behind an explicit confirmation, because a delete can cascade and there's no undo. A read-only connection can never write. ## Access stays scoped Every request runs against your organization only, enforced at the database. The connection acts with your role: operators get no access, viewers stay read-only, the rest read and write. Member emails are never exposed. The same settings page lists every connected client, with its organization and access level, and lets you revoke any of them in one click. Revoking cuts it off immediately. ## What's next Procedure-level scoping (limiting a connection to specific procedures) is on the list. If your agent needs data the server doesn't expose yet, tell us on Discord. ## Roadmap ### Phase & Measurement API URL: https://www.tofupilot.com/roadmap/phase-measurement-api Dedicated API v2 endpoints to create, read, and manage phases and measurements independently from runs. Enables advanced integrations and bulk data operations. ### Station v2 URL: https://www.tofupilot.com/roadmap/station-v2 Lightweight evolution of the Station app: faster deploys, broader platform support, and first-class integration with coding AI agents. ### GitHub Integration URL: https://www.tofupilot.com/roadmap/github-integration Connect GitHub repositories to TofuPilot for automated deployments, test script version control, and CI/CD pipeline integration. ### C# SDK URL: https://www.tofupilot.com/roadmap/c-sdk Integrate TofuPilot with C# test frameworks like OpenTAP. Upload test runs, measurements, and attachments from your .NET test scripts with our C# SDK. ### Self-Service Self-Hosting URL: https://www.tofupilot.com/roadmap/self-service-self-hosting Deploy TofuPilot on your own infrastructure with a free Lab license. Sign in to Orbit, grab your license key, and run the deploy script. No sales call needed. ### Workflow trigger vars URL: https://www.tofupilot.com/roadmap/workflow-trigger-vars Send emails on failed runs with a direct link to the report, and filter triggers on multiple outcomes at once with the "is one of" operator. ### Build Artifacts URL: https://www.tofupilot.com/roadmap/build-artifacts Builds run once in isolation per commit, producing reproducible artifacts. Stations install fast, no local venv or dependency management. ### Dashboard UI v2 URL: https://www.tofupilot.com/roadmap/dashboard-ui-v2 Redesigned dashboard with 10x more filters, a new table UI, global search, and improved navigation to find test data faster. ### SPC Multi-Select URL: https://www.tofupilot.com/roadmap/spc-multi-select Compare multiple measurements on one control chart. Shift+Click to select measurements with matching type, units, and limits. Shared stats and phase filtering. ### Python 3.13 & 3.14 builds URL: https://www.tofupilot.com/roadmap/python-313-314-builds Pick Python 3.12, 3.13, or 3.14 for your test stations from your pyproject.toml. Use the latest language features with no extra setup needed. ### ISO 27001 URL: https://www.tofupilot.com/roadmap/iso-27001 Achieving ISO 27001 certification for enterprise-grade information security management, data protection, and compliance assurance. ### Auto-Push Flow URL: https://www.tofupilot.com/roadmap/auto-push-flow Push to your main branch and TofuPilot automatically deploys the new test script to all linked stations. No manual step. ### Unit Attachments URL: https://www.tofupilot.com/roadmap/unit-attachments Attach files, images, and documents directly to units and test runs. Store evidence, photos, and reports alongside test data. ### Slack Integration URL: https://www.tofupilot.com/roadmap/slack-integration Get real-time alerts in Slack channels on test failures, threshold breaches, and key production events. Stay informed without context switching. ### MATLAB SDK URL: https://www.tofupilot.com/roadmap/matlab-sdk Upload MATLAB test results to TofuPilot with a native connector. Bridge simulation and hardware validation data in a single platform. ### Teams URL: https://www.tofupilot.com/roadmap/teams Segment data visibility across teams, facilities, or external suppliers. Control who sees what with team-based access scoping. ### NI TestStand Plugin URL: https://www.tofupilot.com/roadmap/ni-teststand-plugin Native NI TestStand plugin to upload test runs, measurements, and attachments to TofuPilot directly from your TestStand sequences without custom code. ### Measurements Table URL: https://www.tofupilot.com/roadmap/measurements-table New measurement table in the run page with improved filtering, sorting, and inline limit visualization for faster hardware test analysis. ### Process Control (Insight) URL: https://www.tofupilot.com/roadmap/process-control-insight Rebuilt Insights page with interactive control charts, spec and control limits, histograms, capability indices (Cp, Cpk, Pp, Ppk), and drill-down filters. ### OpenHTF Docs v2.0 URL: https://www.tofupilot.com/roadmap/openhtf-docs-v20 Revamped OpenHTF documentation with clearer step-by-step guides, real-world examples, and best practices for hardware testing. ### Roles & Permissions URL: https://www.tofupilot.com/roadmap/roles-permissions Assign admin, operator, or viewer roles to control who can access, modify, or export hardware test data in your organization. ### TofuPilot Chat URL: https://www.tofupilot.com/roadmap/tofupilot-chat Ask questions about your hardware test data in natural language. Get instant answers, charts, and insights without writing queries. ### SSO URL: https://www.tofupilot.com/roadmap/sso Single sign-on with your identity provider — Okta, Azure AD, OneLogin, and more — for secure, centralized team authentication. ### Guides URL: https://www.tofupilot.com/roadmap/guides Step-by-step tutorials and best practices to help you build, automate, and scale hardware tests with TofuPilot and OpenHTF. ### Phase Pareto URL: https://www.tofupilot.com/roadmap/phase-pareto Rank phases by failures, retries, and wall-clock duration to find the weakest step, then drill into the measurements behind each failure. ### InvenTree Integration URL: https://www.tofupilot.com/roadmap/inventree-integration Sync test data with InvenTree inventory. Upload test results to stock items, update statuses, transfer stock, complete builds, and print labels from workflows. ### Get Started Videos URL: https://www.tofupilot.com/roadmap/get-started-videos Video tutorials to get up and running with TofuPilot quickly — from first test upload to full production line deployment and beyond. ### Rust SDK URL: https://www.tofupilot.com/roadmap/rust-sdk Official Rust client for TofuPilot API v2. Generated from OpenAPI spec with full endpoint coverage, typed responses, integration tests, and crates.io release. ### Discord Integration URL: https://www.tofupilot.com/roadmap/discord-integration Send test result notifications to Discord channels via webhooks. Trigger alerts on test failures, unit creation, or batch events to keep your team informed. ### C++ SDK Client URL: https://www.tofupilot.com/roadmap/c-sdk-client Official C++ client for TofuPilot API v2. Generated from OpenAPI spec with full endpoint coverage, typed responses, and integration tests for native workflows. ### ARM64 Station Support URL: https://www.tofupilot.com/roadmap/arm64-station-support Run TofuPilot stations on Raspberry Pi 4/5 and other ARM64 single-board computers for cost-effective production line testing. ### TofuPilot Agent URL: https://www.tofupilot.com/roadmap/tofupilot-agent AI agent that queries your hardware test data, runs statistical analyses, generates reports, and takes actions on your behalf. ### Station URL: https://www.tofupilot.com/roadmap/station Headless production test runner with offline mode, local caching, and automatic sync — built to run tests on the manufacturing line. ### Alerts URL: https://www.tofupilot.com/roadmap/alerts Define rules on metrics like FPY, failure rate, Cpk, or station heartbeat. Route triggered alerts to email, Slack, or webhook channels. ### Linear Integration URL: https://www.tofupilot.com/roadmap/linear-integration Connect TofuPilot to Linear to automatically create issues and add comments from test results. Trigger workflows on failures to open bugs or track tasks. ### Workflows URL: https://www.tofupilot.com/roadmap/workflows Automate actions when test events occur. Connect to Odoo, Linear, InvenTree, Discord, Slack, or any HTTP API. Filter, branch, and chain steps visually. ### GitLab Integration URL: https://www.tofupilot.com/roadmap/gitlab-integration Connect GitLab repositories with real-time webhook sync, soft delete for branches and commits, and support for both Group and Personal Access Tokens. ### Operator UI advanced mode URL: https://www.tofupilot.com/roadmap/operator-ui-advanced-mode Improved Operator UI for line technicians with deeper run details, richer phase context, and faster troubleshooting on the shop floor. ### Desktop Remote Control URL: https://www.tofupilot.com/roadmap/desktop-remote-control Remotely control station desktops from TofuPilot Studio for debugging, configuration, live test monitoring, and troubleshooting. ### Changelog URL: https://www.tofupilot.com/roadmap/changelog Stay up to date with every new feature, improvement, bug fix, and product release shipped by the TofuPilot engineering team. ### Golden & Black Samples URL: https://www.tofupilot.com/roadmap/golden-black-samples Mark units as golden or black samples for calibration, validation, and SPC reference tracking across your test stations. ### Instrument identity URL: https://www.tofupilot.com/roadmap/instrument-identity Persistent plug instances keyed by serial. Auto-create on first identify(), link logs and measurements, track firmware and calibration across runs. ### SCIM Provisioning URL: https://www.tofupilot.com/roadmap/scim-provisioning Automatically sync users and groups from your identity provider to TofuPilot. Onboard and offboard team members instantly. ### Instant Rollback URL: https://www.tofupilot.com/roadmap/instant-rollback Roll back every station to a previous test script build in one click. No rebuild, and auto-push pauses until you push again or resume it. ### MCP Server URL: https://www.tofupilot.com/roadmap/mcp-server Official MCP server for TofuPilot. Lets AI assistants like Claude query test data, manage runs, units, and stations through the Model Context Protocol. ### Kubernetes deployment URL: https://www.tofupilot.com/roadmap/kubernetes-deployment Deploy TofuPilot on your Kubernetes cluster with a Helm chart, ready for high availability, scaling, and enterprise-grade operations. ### OAuth2 SMTP Support URL: https://www.tofupilot.com/roadmap/oauth2-smtp-support Add OAuth2 (XOAUTH2) authentication for SMTP email in self-hosted deployments. Required for Microsoft 365 compatibility after Basic Auth deprecation. ### C# API Client URL: https://www.tofupilot.com/roadmap/c-api-client Official C# client for TofuPilot API v2. Auto-generated from OpenAPI spec with full endpoint coverage, typed responses, 131 xUnit tests, and NuGet publishing. ### Docs 2.0 URL: https://www.tofupilot.com/roadmap/docs-20 Revamped documentation with restructured pages, interactive guides, updated API reference, and improved search across all content. ### Templates URL: https://www.tofupilot.com/roadmap/templates Ready-to-use Python test templates for PCB, battery, motor, and sensor testing. Copy, customize, and deploy to your stations in minutes. ### ARMv7 Station Support URL: https://www.tofupilot.com/roadmap/armv7-station-support Run TofuPilot stations on ARMv7 devices like BeagleBone and industrial SBCs for embedded and legacy production hardware testing. ### Run metadata URL: https://www.tofupilot.com/roadmap/run-metadata Attach custom key-value metadata to runs beyond built-in fields, for traceability, filtering, and export across procedures and stations. ### SAP Integration URL: https://www.tofupilot.com/roadmap/sap-integration Sync hardware test data and quality metrics with SAP ERP and QM modules. Bridge manufacturing execution and quality management systems. ### Import test files URL: https://www.tofupilot.com/roadmap/import-test-files Import existing test reports into TofuPilot from OpenHTF, WATS, ATML, NI TestStand, STDF, CSV, and Excel files, with downloadable sample files for every format. ### Unit metadata URL: https://www.tofupilot.com/roadmap/unit-metadata Attach custom key-value metadata to units beyond built-in fields (serial, part, revision, batch), queryable and exportable across runs. ### Teams Integration URL: https://www.tofupilot.com/roadmap/teams-integration Get real-time alerts in Microsoft Teams on test failures, threshold breaches, and key production events. Stay informed without switching tools. ### LabVIEW SDK URL: https://www.tofupilot.com/roadmap/labview-sdk LabVIEW VIs wrapping the TofuPilot API. Create runs, upload measurements with pass/fail limits, and log results from your LabVIEW test sequences. ### Bitbucket Integration URL: https://www.tofupilot.com/roadmap/bitbucket-integration Connect Bitbucket repositories to TofuPilot for automated deployments, test script version control, and CI/CD pipeline integration. ### Vision AI Block URL: https://www.tofupilot.com/roadmap/vision-ai-block Visual inspection block in Studio using AI-powered defect detection. Catch cosmetic defects, misalignments, and assembly errors. ### pytest Connector URL: https://www.tofupilot.com/roadmap/pytest-connector Upload pytest test results to TofuPilot with a native connector. Analyze hardware validation tests alongside production data in one view. ### Plug actions URL: https://www.tofupilot.com/roadmap/plug-actions Trigger plug methods from UI component actions so operators can drive plugs from a switch or button, with results captured in phase_results. ### Custom Validators URL: https://www.tofupilot.com/roadmap/custom-validators Define your own pass/fail logic beyond fixed operators. Bring validation rules from your sequencer, or compose custom limits per measurement type. ### Station touch mode URL: https://www.tofupilot.com/roadmap/station-touch-mode Touch-optimized Station UI for production screens with touch input, enabling operators to run procedures without mouse or keyboard. ### S-Parameters & Masks URL: https://www.tofupilot.com/roadmap/s-parameters-masks Store complex waveforms as magnitude and phase, then validate both concurrently against per-point masks for RF and SerDes characterization. ### Shared phases URL: https://www.tofupilot.com/roadmap/shared-phases Reuse a full phase block including its ui: definition from a shared YAML file, so operators avoid duplicating UI across procedures. ### Dev stations URL: https://www.tofupilot.com/roadmap/dev-stations Flag stations as dev or production: run preview builds on a dev bench safely, and keep development runs out of production KPIs and alerts. ### OpenHTF in Station v2.0 URL: https://www.tofupilot.com/roadmap/openhtf-in-station-v20 Run, debug, and monitor OpenHTF tests directly from TofuPilot Studio with live logs, breakpoints, and result inspection. ### SOC 2 URL: https://www.tofupilot.com/roadmap/soc-2 SOC 2 Type II compliance for enterprise-grade security, availability, and confidentiality. Built for teams in regulated industries. ### Jira Integration URL: https://www.tofupilot.com/roadmap/jira-integration Create Jira issues automatically from test failures and sync status updates between TofuPilot and your project management workflow. ### Robot Framework URL: https://www.tofupilot.com/roadmap/robot-framework Upload Robot Framework test results to TofuPilot with a native connector. Track hardware test trends and quality metrics over time. ### Repair URL: https://www.tofupilot.com/roadmap/repair Log failures and repair actions on failing units, with AI-suggested failure and repair codes plus rework yield and repair-effectiveness analytics. ### News URL: https://www.tofupilot.com/roadmap/news Blog posts, product updates, engineering deep-dives, and company announcements — all in one place to keep your team informed. ### Odoo Integration URL: https://www.tofupilot.com/roadmap/odoo-integration Sync hardware test data and quality metrics with your Odoo ERP instance. Bridge manufacturing execution and quality management systems. ### Reports URL: https://www.tofupilot.com/roadmap/reports Build custom reports combining multiple insights, charts, and metrics to visualize hardware test data across your organization. ### Waveform Persistence URL: https://www.tofupilot.com/roadmap/waveform-persistence Overlay waveforms from all runs as an intensity map, eye-diagram style, with the current run drawn on top to spot degradation trends at a glance. ### Environment Variables URL: https://www.tofupilot.com/roadmap/environment-variables Manage environment variables per station for deployments, secrets, and configuration. Keep credentials and machine-specific configuration out of your test code. ### Process change markers URL: https://www.tofupilot.com/roadmap/process-change-markers Mark process changes on control charts with dated annotations and filter analytics to before, after or between changes. ### Sticky Operator Inputs URL: https://www.tofupilot.com/roadmap/sticky-operator-inputs Mark operator input fields as sticky so their values persist across runs when using Run Again, reducing repetitive entry for recurring tests. ## Questions & Answers ### Is TofuPilot station licensing tied to hardware? URL: https://www.tofupilot.com/questions/is-tofupilot-station-licensing-tied-to-hardware A station counts against your plan when it is registered, not by hardware fingerprint. You can remove a station and register another in its place, so replacing a bench PC or reflashing a machine does not permanently consume a station slot. ### When am I billed for TofuPilot? URL: https://www.tofupilot.com/questions/when-am-i-billed-for-tofupilot Pro plans bill monthly through Stripe based on selected options and the previous month's usage. Usage is visible under Settings > Usage with notifications at 75% and 100% of included quotas. Enterprise plans offer monthly or annual invoicing. ### How much does TofuPilot cost? URL: https://www.tofupilot.com/questions/how-much-does-tofupilot-cost Three plans. Lab is free: 1 seat, 1 station, 100 runs per month, 10 GB storage. Pro is $50 per user per month with 1,000 runs per month included (then $0.01 per run), 100 GB storage (then $0.10 per GB), and 1 station included (then $50 per station). Enterprise adds SSO/SCIM at scale, self-hosting, air-gapped operation, custom limits, and SLA-backed support. ### Is TofuPilot subject to the US Patriot Act or CLOUD Act? URL: https://www.tofupilot.com/questions/is-tofupilot-subject-to-the-us-patriot-act-or-cloud-act No. TofuPilot SA is a Swiss-incorporated company with no US legal entity. US surveillance laws, including the CLOUD Act, FISA Section 702, and National Security Letters, cannot compel TofuPilot to disclose customer data. For customers requiring full infrastructure sovereignty, TofuPilot offers self-hosted deployment on your own servers. ### How long does integration take? URL: https://www.tofupilot.com/questions/how-long-does-integration-take Most teams are up and running within 30 minutes. Integration requires a single line of Python — no migration or infrastructure changes needed. ### Which test frameworks are supported? URL: https://www.tofupilot.com/questions/which-test-frameworks-are-supported TofuPilot integrates natively with OpenHTF via an open-source plugin, and supports pytest, Robot Framework, and custom scripts through the Python client. ### How is test data protected? URL: https://www.tofupilot.com/questions/how-is-test-data-protected All data is encrypted at rest and in transit. TofuPilot SA is Swiss-incorporated, so your data is protected by Swiss data protection law and is outside the reach of US surveillance legislation such as the CLOUD Act. Organizations requiring full control can self-host TofuPilot on-premise or in air-gapped environments. ### Does TofuPilot scale for high-volume manufacturing? URL: https://www.tofupilot.com/questions/does-tofupilot-scale-for-high-volume-manufacturing TofuPilot processes thousands of test runs daily and supports real-time queries across millions of data points, backed by a 99.9% uptime SLA. ### Can TofuPilot be self-hosted? URL: https://www.tofupilot.com/questions/landing-can-tofupilot-be-self-hosted Yes. Self-hosted deployment is available on the Enterprise plan, providing complete data sovereignty for regulated and air-gapped environments. ### Do you support air-gapped or offline environments? URL: https://www.tofupilot.com/questions/landing-what-about-air-gapped-or-offline-environments The self-hosted deployment operates fully offline with no external dependencies. Data synchronizes automatically when connectivity is restored. ### Can instrument connections stay open across an entire unit or station session? URL: https://www.tofupilot.com/questions/can-instrument-connections-stay-open-across-an-entire-unit-or-station-session Yes. Plugs have a lifecycle scope: phase, slot, run, or station. With scope: station, the plug class is constructed once in a long-lived Python process, its connections stay open across units until the station stops, and only the first unit after startup pays the connection cost. Every test method reuses the same live object, so instrument sessions are never reopened between phases or between units. ### What is TofuPilot's relationship with OpenHTF? URL: https://www.tofupilot.com/questions/what-is-tofupilots-relationship-with-openhtf We've used TofuPilot for years on drone manufacturing floors and maintain openhtf.org, the open-source OpenHTF documentation site. We're also close to the OpenHTF founders. TofuPilot provides native OpenHTF integration that requires just one line of code to start streaming test results with full metadata preservation. ### Can I sync TofuPilot test failures to Linear? URL: https://www.tofupilot.com/questions/can-i-sync-tofupilot-test-failures-to-linear Yes. The Linear integration lets failing runs and phases open or update tickets from the dashboard. It offers two actions: Create issue (with optional assignee, labels, priority, and estimate in story points) and Add comment. Connect via OAuth under Settings > Integrations > Linear; no API keys are required, and you can connect multiple teams by repeating the flow. Text fields support {{variable}} interpolation, and the created issue's ID and URL are exposed to downstream workflow nodes. ### How do I migrate from NI TestStand to TofuPilot? URL: https://www.tofupilot.com/questions/how-do-i-migrate-from-ni-teststand-to-tofupilot Export an NI TestStand XML report (.xml) and create a procedure in the dashboard. Drag the .xml onto the import page and click Import; TofuPilot auto-detects the file by its root element. Each becomes a run with its UUTResult as the outcome, ResultList steps become phases, and numeric or pass-fail limit steps become measurements with limits. Via API, upload then import with importer TESTSTAND (up to 100 files per request). ### How do I install the TofuPilot Python SDK? URL: https://www.tofupilot.com/questions/how-do-i-install-the-tofupilot-python-sdk Install from PyPI: pip install tofupilot. Import it as "from tofupilot.v2 import TofuPilot". Pass your API key to the constructor or set the TOFUPILOT_API_KEY environment variable. The type-safe SDK wraps the REST API and supports creating runs with measurements, phases, and validators, cursor-paginated run listing, attachment upload and download, and async methods. ### Can TofuPilot sync test results to InvenTree? URL: https://www.tofupilot.com/questions/can-tofupilot-sync-test-results-to-inventree Yes. The InvenTree integration writes TofuPilot test data into your InvenTree instance so build outputs, stock items, and test reports stay in sync. Five actions are available: upload a test result, update stock status, transfer stock between locations, complete a build output, and print a label. Connect with an API token that inherits the generating user's permissions, plus the base URL, configured once per instance. ### Can TofuPilot alert me when a measurement drifts or a unit fails? URL: https://www.tofupilot.com/questions/can-tofupilot-alert-me-when-a-measurement-drifts-or-a-unit-fails Yes. Workflows trigger on run events and send notifications through Slack, Discord, email, or HTTP calls, and AI alerts watch for anomalies such as measurement drift or yield drops. ### Can I query my TofuPilot test data in natural language? URL: https://www.tofupilot.com/questions/can-i-query-my-tofupilot-test-data-in-natural-language Yes, with TofuPilot Chat. It answers questions across every run, unit, part, batch, and station in your organization. Chat calls typed tools backed by the REST API, so every answer is grounded in real data with links to the dashboard. It handles aggregations ("How many runs failed yesterday?"), drill-downs into failing phases or measurements, comparisons across revisions, stations, or batches, and statistics like the Cpk of a measurement over the last 1000 runs. You only see data your role and team scope allow. ### What is TofuPilot Agent? URL: https://www.tofupilot.com/questions/what-is-tofupilot-agent TofuPilot Agent runs autonomous root cause analysis on failing units inside the chat interface. It frames the failure in plain language, hypothesizes candidate causes across phase, measurement, station, revision, and operator, gathers evidence by querying run and measurement data, scores hypotheses with Wilson 95% confidence intervals against baselines, then ranks and reports the highest-confidence causes with inline evidence. Use it to confirm whether a cause is the part, the station, or the test itself; for simple lookups use Chat. Results can be saved as shareable links or attached to tickets. ### Can I attach files to a test run in TofuPilot? URL: https://www.tofupilot.com/questions/can-i-attach-files-to-a-test-run-in-tofupilot Yes. An attachment is a binary file persisted alongside a run, such as a photo, a scope capture, a diagnostic dump, or a PDF report. Each attachment belongs to one run and can optionally be linked to the phase that produced it. Upload attachments through the REST API or the Python SDK. ### How do I run a test procedure with the TofuPilot CLI? URL: https://www.tofupilot.com/questions/how-do-i-run-a-test-procedure-with-the-tofupilot-cli Run tofupilot run [PATH], where PATH is an optional procedure .yaml file, directory, or Python entry point. Without a path it runs locally and, without --upload, never contacts the dashboard. Use --deployment to run a pulled deployment, --upload to sync a local run to the dashboard, --tui/--no-tui for the terminal operator UI, and --kiosk/--no-kiosk for the browser kiosk UI. ### What is a sub-unit in TofuPilot? URL: https://www.tofupilot.com/questions/what-is-a-sub-unit-in-tofupilot A sub-unit is a child unit assembled into a parent unit, such as a PCB inside a finished product. A parent unit can reference multiple sub-units by their child serial numbers, enabling full genealogy tracking of nested components like PCBs, batteries, and enclosures. This lets you trace which components went into which assembly, aligning with MES genealogy records and IPC-1782 traceability. ### What's the difference between a part, revision, and unit? URL: https://www.tofupilot.com/questions/whats-the-difference-between-a-part-revision-and-unit A part is the product or component family that units belong to (e.g. PCB-V1). A revision is a version of a part; it defaults to A. A unit is the individual physical item under test, identified by a serial number and belonging to one part-and-revision combination. Batches are an optional layer grouping units made in the same lot. The model maps to PLM and ERP item masters. ### How do I install TofuPilot on Linux? URL: https://www.tofupilot.com/questions/how-do-i-install-tofupilot-on-linux Run the install script in a terminal: curl -fsSL https://tofupilot.sh/install | sh. This installs the tofupilot CLI. To register a machine as a station, authenticate headlessly with a one-hour setup token from the dashboard: tofupilot login --token . The station daemon runs under systemd; manage it with tofupilot service start|stop|status. ### How do I uninstall TofuPilot? URL: https://www.tofupilot.com/questions/how-do-i-uninstall-tofupilot Run tofupilot uninstall. It removes the CLI binary and, by default, every local data file including queued uploads, pulled deployments, and cached config. Use --keep-data to preserve run data, queued uploads, and pulled deployments before reinstalling, and --yes to skip the confirmation prompt in CI. ### What can I do with the TofuPilot CLI? URL: https://www.tofupilot.com/questions/what-can-i-do-with-the-tofupilot-cli The CLI has 26+ subcommands. Common ones: tofupilot login/logout/whoami (auth), tofupilot run (execute a procedure locally with the operator UI), tofupilot deploy (build and deploy procedures to the cloud), tofupilot pull (pull deployments), tofupilot runs/batches/units/measurements (manage test data), tofupilot stations (manage factory stations), tofupilot service (run as a system daemon), and tofupilot update/uninstall. Every command accepts the global --json flag for machine-readable output. ### Can TofuPilot connect to my ERP (Odoo)? URL: https://www.tofupilot.com/questions/can-tofupilot-connect-to-my-erp-odoo Yes. The Odoo integration pushes test data into your Odoo instance so production records, quality checks, and chatter stay in sync. Three actions are available: Create record on any Odoo model (e.g. product.product, quality.check, mrp.production), Update record by ID, and Post message to a record's chatter thread. Setup requires an Odoo user with model access, an API key, plus the Odoo URL, database name, and login. ### What is a batch in TofuPilot? URL: https://www.tofupilot.com/questions/what-is-a-batch-in-tofupilot A batch groups units you produced together, such as a manufacturing lot, a build day, or a contract manufacturer shipment. Its identity is its number; all other information is derived from its linked units. Units link to a batch at creation via the batch_number parameter or afterward by manual assignment. Batch numbers are 1-60 characters matching ^[a-zA-Z0-9_.:+-]+$, unique per organization (case-insensitive). Deleting a batch keeps unit data but removes the assignment, and empty batches are cleaned up automatically. ### How do I deploy a procedure with the TofuPilot CLI? URL: https://www.tofupilot.com/questions/how-do-i-deploy-a-procedure-with-the-tofupilot-cli Run tofupilot deploy from a directory linked to a procedure that contains a pyproject.toml. It uploads the local source tree, builds it in the cloud, and streams build logs. By default it creates a preview deployment; use tofupilot deploy --prod to target production, after which every linked station picks it up between runs. Deploy requires a user login; station setup tokens cannot deploy code. ### How do I import OpenHTF results into TofuPilot? URL: https://www.tofupilot.com/questions/how-do-i-import-openhtf-results-into-tofupilot Save an OpenHTF test record as JSON via the OutputToJSON callback and create a procedure in the dashboard. Drag the .json file to the dropzone (auto-detected as OpenHTF) and click Import; no column mapping is needed. The record becomes a run, dut_id becomes the unit serial number, phases become phases, measurements with validators become measurements with limits, and outcome sets the run outcome. Via API, upload then import by upload_id with the OPENHTF importer at POST /v2/import. ### Can TofuPilot call any external API over HTTP? URL: https://www.tofupilot.com/questions/can-tofupilot-call-any-external-api-over-http Yes. The HTTP integration lets a workflow send a request to any URL, so TofuPilot events can trigger systems without a native integration. It supports GET, POST, PUT, PATCH, and DELETE, with {{variable}} interpolation in URLs, headers, and bodies. Downstream nodes can read the response status, body, and headers. Limits: timeout 1-60 seconds (default 10), request and response bodies max 1 MB each, and up to three retries with exponential backoff. ### Does TofuPilot have an MCP server for AI agents? URL: https://www.tofupilot.com/questions/does-tofupilot-have-an-mcp-server-for-ai-agents Yes. The TofuPilot MCP server lets AI agents (Claude, Cursor, and any Model Context Protocol client) work with your test data. It exposes read tools by default (list and retrieve runs, units, parts, batches, procedures, stations, users, logs, phases, measurements, deployments) and optional write tools. Connect Claude with: claude mcp add --transport http tofupilot https://www.tofupilot.app/mcp, then run /mcp to authorize in your browser via OAuth. Each client binds to one organization with scoped read or write access. ### What happens if a long test run crashes near the end? URL: https://www.tofupilot.com/questions/what-happens-if-a-long-test-run-crashes-near-the-end Phases are isolated on worker processes: if one dies, the engine replaces it and execution continues instead of taking the whole run down, and retries are configurable per phase. Plug processes are health-checked between units and restarted automatically if they died, so a crash never carries into the next unit. If the station loses the network, results are stored locally and uploaded when connectivity returns. This matters most on runs measured in hours, where losing the run means losing a full shift of bench time. ### Can an MES block a unit before the test runs? URL: https://www.tofupilot.com/questions/can-an-mes-block-a-unit-before-the-test-runs Yes, in both directions. The MES check runs as the first step of the execution, before any test phase: if the MES refuses the unit, the run stops there and the dashboard shows a dedicated status, so a unit blocked by the MES is never counted as a unit that failed testing. At the end of the run the results are sent back to the MES the same way. The check is a normal API call from your own code, so it works with whichever system you run, and dashboard-side MES and ERP integrations are available for the synchronization that does not need to gate execution. ### Does the Rust engine run each Python test in a separate process? URL: https://www.tofupilot.com/questions/does-the-rust-engine-run-each-python-test-in-a-separate-process Two mechanisms. Plugs run as persistent processes, one per plug instance, addressed over a local TCP socket, so instrument objects and their state stay alive across phases. Test phases run on a pool of worker processes, which is what isolates a crashing phase from the rest of the run. Both are standard Python subprocesses, not an embedded interpreter, and the environment and dependencies are managed automatically with uv. ### How many units can a station test in parallel? URL: https://www.tofupilot.com/questions/how-many-units-can-a-station-test-in-parallel Parallel slots are native to the execution engine, which is why it is written in Rust. Configurations of a few slots up to several dozen are normal operation, including burn-in and aging testers where units simply stay powered for hours. Each slot keeps its own serial number, part number and revision. Slots do not require a live instrument connection, so a large oven load can be identified and recorded without a per-unit script. Very large configurations, in the hundreds of slots, have no architectural limit but are worth validating on your actual bench. ### How do I integrate OpenHTF with TofuPilot? URL: https://www.tofupilot.com/questions/how-do-i-integrate-openhtf-with-tofupilot Deployed through the TofuPilot CLI, OpenHTF scripts stream live test data and the operator UI with zero configuration. Standalone, it takes one wrapper: import TofuPilot from tofupilot.openhtf, then wrap your test with "with TofuPilot(test): test.execute()". All phases, measurements, and attachments upload automatically. ### How do I record the operator of a test in TofuPilot? URL: https://www.tofupilot.com/questions/how-do-i-record-the-operator-of-a-test-in-tofupilot Pass the operated_by field (operator email) when creating the run through the API or SDK. The run then shows the operator identity in the dashboard, separate from the station that uploaded it. ### Do test stations need permanent internet access? URL: https://www.tofupilot.com/questions/do-test-stations-need-permanent-internet-access No. Every completed run is persisted to a local queue before upload; a background loop drains it every few seconds and retries with backoff from 15 seconds up to a 1 hour cap, so network outages lose no data. Inspect or replay the queue with commands like "tofupilot list-queue", "tofupilot queue-retry", and "tofupilot queue-export". Fully offline factories run the self-hosted deployment instead. ### How do I install TofuPilot on macOS? URL: https://www.tofupilot.com/questions/how-do-i-install-tofupilot-on-macos Run curl -fsSL https://tofupilot.sh/install | sh in Terminal to install the tofupilot CLI (same command as Linux). Register a station headlessly with tofupilot login --token , which places the daemon under launchd, managed via tofupilot service start|stop|status. ### How do I install the TofuPilot CLI? URL: https://www.tofupilot.com/questions/how-do-i-install-the-tofupilot-cli Run the install script on Linux or macOS: curl -fsSL https://tofupilot.sh/install | sh. A PowerShell variant is available for Windows. After installing, run "tofupilot login", then "tofupilot run" in your test project to execute and stream your first test. ### Does TofuPilot run on Windows? URL: https://www.tofupilot.com/questions/does-tofupilot-run-on-windows Yes. The CLI, Station, and Studio run on Windows through a PowerShell installer. Linux and macOS are also fully supported. ### How do I migrate an existing test suite to TofuPilot without downtime? URL: https://www.tofupilot.com/questions/how-do-i-migrate-an-existing-test-suite-to-tofupilot-without-downtime TofuPilot supports incremental migration: migrate one product line at a time (vertical), one test stage at a time (horizontal), or a mix (hybrid). Your legacy system keeps running while migrated stations stream to TofuPilot, so production never stops. ### Does TofuPilot work with OpenHTF, pytest, and Robot Framework? URL: https://www.tofupilot.com/questions/does-tofupilot-work-with-openhtf-pytest-and-robot-framework Yes. TofuPilot supports OpenHTF, pytest, Robot Framework, and its own open-source TofuPilot framework. Any custom framework can integrate through the REST API or one of the SDKs (Python, C#, Rust, C++, MATLAB). ### Can I upload OpenHTF results that were recorded offline? URL: https://www.tofupilot.com/questions/can-i-upload-openhtf-results-that-were-recorded-offline Yes. Write the OpenHTF JSON report to disk, then upload it later with client.create_run_from_openhtf_report("./report.json"). Original timestamps are preserved. Stations running the CLI also queue completed runs locally and upload automatically when connectivity returns. ### How do I add a TofuPilot desktop shortcut to a station? URL: https://www.tofupilot.com/questions/how-do-i-add-a-tofupilot-desktop-shortcut-to-a-station In the dashboard, go to Stations, select the station, open Setup, and toggle "Desktop shortcut" on. The CLI writes a launcher to the station's desktop equivalent to running tofupilot run in the station's working directory, launching whichever operator UI the station uses. The file is TofuPilot.desktop on Linux, TofuPilot.command on macOS, and TofuPilot.lnk on Windows. Toggling the setting off deletes the shortcut. ### Can I use pytest for hardware testing with TofuPilot? URL: https://www.tofupilot.com/questions/can-i-use-pytest-for-hardware-testing-with-tofupilot Yes. The CLI runs pytest suites natively under an embedded plugin: each test_* function becomes a phase, and assert statements like "assert lo <= x <= hi" become numeric measurements with limits. Run identity is read from [tool.tofupilot] in pyproject.toml. ### Can I create TofuPilot test phases dynamically at runtime? URL: https://www.tofupilot.com/questions/can-i-create-tofupilot-test-phases-dynamically-at-runtime In the TofuPilot framework, phases are declared in a YAML procedure and each phase runs a Python function. If you need programmatic phase generation at runtime, use OpenHTF, where phases are plain Python callables you can build dynamically. ### Can I share data between test phases in TofuPilot? URL: https://www.tofupilot.com/questions/can-i-share-data-between-test-phases-in-tofupilot Yes. A phase can read an earlier phase's results by declaring a function parameter named after that phase's key; TofuPilot injects a read-only result object exposing its measurements, outcome, and duration. Use depends_on to guarantee the earlier phase runs first. ### Are TofuPilot measurements scoped to a test phase? URL: https://www.tofupilot.com/questions/are-tofupilot-measurements-scoped-to-a-test-phase Measurements belong to the phase that records them. Each measurement is a typed value (number, string, boolean, JSON, or multi-dimensional series) validated against limits; a failing validator fails the measurement, its phase, and the run. ### Can I control lab instruments over SCPI, VISA, or LabVIEW? URL: https://www.tofupilot.com/questions/can-i-control-lab-instruments-over-scpi-visa-or-labview Python is the primary scripting language, so instrument control works through standard libraries like PyVISA, pyserial, or pymodbus inside plugs, which hold persistent hardware connections. Phases can also call external binaries in any language through an executable phase. LabVIEW and TestStand environments integrate through the SDKs. ### Which operating systems does TofuPilot support? URL: https://www.tofupilot.com/questions/which-operating-systems-does-tofupilot-support The TofuPilot CLI supports Linux, macOS, and Windows. On macOS and Linux, install with curl -fsSL https://tofupilot.sh/install | sh; on Windows, run the PowerShell installer from https://tofupilot.sh/install.ps1. The station daemon runs under systemd on Linux and launchd on macOS. ### How do I create a test run with the TofuPilot API? URL: https://www.tofupilot.com/questions/how-do-i-create-a-test-run-with-the-tofupilot-api Use the Python SDK (client.runs.create with procedure_id, serial_number, part_number, outcome, and phases) or the REST endpoint POST https://www.tofupilot.app/api/v2/runs. Authenticate with an API key passed as a bearer token or the TOFUPILOT_API_KEY environment variable. ### How do I update the TofuPilot CLI? URL: https://www.tofupilot.com/questions/how-do-i-update-the-tofupilot-cli Run tofupilot update. It checks for a newer CLI release and installs it in place. The CLI also checks in the background and prompts you when an update is available. ### How do I create and manage API keys? URL: https://www.tofupilot.com/questions/how-do-i-create-and-manage-api-keys Open Settings > API keys, click Create key, name it, and pick an expiry. The key is shown once. There are two types: user keys, which inherit your role and team scope, and station keys, which are issued at station registration and scoped to that station. Revocation is immediate. ### Does TofuPilot integrate with Slack? URL: https://www.tofupilot.com/questions/does-tofupilot-integrate-with-slack Yes. TofuPilot posts a message to a Slack channel whenever a workflow fires, using a Slack incoming webhook (no OAuth app or token management). Create an incoming webhook in Slack, copy the URL, then add a "Send Slack message" action to a workflow. The message is plain text and supports {{variable}} interpolation; you can override the bot name and icon, and chain multiple Slack actions to reach several channels. ### Can two units share a serial number if they have different part numbers? URL: https://www.tofupilot.com/questions/can-two-units-share-a-serial-number-if-they-have-different-part-numbers No. Serial numbers are unique per organization (case-insensitive), so a unit's full history stays unambiguous. If your lines reuse short serials across products, send the part number explicitly alongside the serial, or embed it in the serial (for example PCB01-000123). ### Can I edit TofuPilot run metadata after upload? URL: https://www.tofupilot.com/questions/can-i-edit-tofupilot-run-metadata-after-upload Yes. The v2 endpoint PATCH /v2/runs/{id}/metadata upserts custom metadata key-value pairs on a run after ingestion. To correct a unit's serial number or part number, edit the unit inline rather than using this endpoint. ### How do I fetch a unit's latest test run via the TofuPilot API? URL: https://www.tofupilot.com/questions/how-do-i-fetch-a-units-latest-test-run-via-the-tofupilot-api Query runs.list filtered by serial number, sorted by start time descending with limit=1. For per-unit lifetime views in the dashboard, use the Unit Explorer. ### Does TofuPilot have a REST API without the SDK? URL: https://www.tofupilot.com/questions/does-tofupilot-have-a-rest-api-without-the-sdk Yes. The full platform is exposed as a REST API at https://www.tofupilot.app/api/v2 with API-key authentication, pagination, and typed error codes. Every resource (runs, units, parts, procedures, stations, measurements, phases, logs, attachments, imports) has documented endpoints. ### Can one test station run multiple procedures? URL: https://www.tofupilot.com/questions/can-one-test-station-run-multiple-procedures Yes. Deploy each procedure to the station; the operator then selects which procedure to run from the station UI. A station can hold as many procedures as your workflow needs. ### Can stations run headless, for example in CI or a rack with no display? URL: https://www.tofupilot.com/questions/can-stations-run-headless-for-example-in-ci-or-a-rack-with-no-display Yes. Stations authenticate with long-lived station keys instead of interactive login. The install command embeds a single-use setup token (valid one hour, scoped to one station) that the CLI exchanges for a station key on first boot, so no display or manual sign-in is needed. ### How do I install system packages (non-Python dependencies) on a station? URL: https://www.tofupilot.com/questions/how-do-i-install-system-packages-non-python-dependencies-on-a-station Install native system dependencies on the machine's system Python, then enable the "System packages" toggle in the station's Setup. On every deployment the CLI rebuilds the virtual environment with uv using --system-site-packages, so the venv can see those machine-level packages instead of losing them on each pull. ### How do I launch a TofuPilot station on boot? URL: https://www.tofupilot.com/questions/how-do-i-launch-a-tofupilot-station-on-boot Use the launch-on-boot setting; the CLI installs a system service (systemd on Linux) so the station starts with the machine, including root-level system services for kiosk deployments. ### Can I roll back a bad deployment in TofuPilot? URL: https://www.tofupilot.com/questions/can-i-roll-back-a-bad-deployment-in-tofupilot Yes. Open the deployment's menu in the dashboard and select Instant Rollback to revert stations to any previous deployment immediately. Rollback is dashboard-only for now; API and SDK support is planned. Rolling rollouts let you push a new build one station at a time before rolling it out fleet-wide. ### How do I import test results from a CSV file? URL: https://www.tofupilot.com/questions/how-do-i-import-test-results-from-a-csv-file You need a CSV with a header row and a procedure created in the dashboard. Drop the .csv on the procedure's import page, click the row to open the mapping sidebar, map columns or apply a saved preset, then click Import. Four fields are required: serial number, outcome, started at, and ended at. Duplicate files are flagged by content hash, so re-importing the same CSV links to the existing run. ### Does TofuPilot integrate with Discord? URL: https://www.tofupilot.com/questions/does-tofupilot-integrate-with-discord Yes. TofuPilot posts a message to a Discord channel whenever a workflow fires, via a Discord webhook, so you do not need to host a bot. In Discord, create a webhook under channel Settings > Integrations > Webhooks, copy the URL, then add a "Send Discord message" action to your workflow. The message is plain text with markdown and {{variable}} support, and you can set a custom username and avatar. ### What file formats can I import into TofuPilot? URL: https://www.tofupilot.com/questions/what-file-formats-can-i-import-into-tofupilot TofuPilot imports nine formats in two categories. Tabular (require field mapping): CSV and Excel (.xlsx). Structured (auto-detected): STDF, ATDF, ATML, NI TestStand XML, OpenHTF JSON, WATS WSXF, and WSJF. Each file is parsed into one or more runs through the same path a live test uses, so imported runs are indistinguishable from station-produced ones. ### How do I import an Excel file into TofuPilot? URL: https://www.tofupilot.com/questions/how-do-i-import-an-excel-file-into-tofupilot You need an Excel workbook (.xlsx) with a header row and a procedure created in the dashboard. Drop the .xlsx on the import page, click the row to open the mapping sidebar, apply a saved preset or map columns by hand, then click Import. Only serial number, outcome, started at, and ended at are required; optional mappings include part number, revision number, batch number, operated by, and description. Save a finished mapping as a preset so the next workbook maps itself. ### What analytics does TofuPilot provide out of the box? URL: https://www.tofupilot.com/questions/what-analytics-does-tofupilot-provide-out-of-the-box First pass yield, Cpk and control charts (measurement control), phase Pareto by failure rate and duration, run and unit explorers with filtering, and log search across runs. Charts work on any uploaded measurement without extra configuration. ### Why does my Cpk drop when tests correctly catch failing units? URL: https://www.tofupilot.com/questions/why-does-my-cpk-drop-when-tests-correctly-catch-failing-units Cpk measures process capability, so far-out-of-limit readings from a known upstream failure distort it. Use yield to track pass/fail performance and Cpk to track drift on healthy processes. For measurements that are invalid because an earlier step failed, record no value (None) instead of a garbage reading so it is excluded from capability statistics. ### Can I export test data to CSV or JSON? URL: https://www.tofupilot.com/questions/can-i-export-test-data-to-csv-or-json Yes. The Run Explorer's Export button writes the current filtered run set to CSV (one row per run) or JSON (nested), for all visible rows or a selection. Exports honor team scope and role permissions. For automated pipelines, use the REST API or SDKs. ### How do I install TofuPilot behind a corporate proxy or firewall? URL: https://www.tofupilot.com/questions/how-do-i-install-tofupilot-behind-a-corporate-proxy-or-firewall Allowlist tofupilot.sh on your proxy so the install script can download. Stations only need outbound HTTPS to TofuPilot; no inbound ports are required. If the proxy enforces per-user session limits, use per-user proxy credentials rather than shared ones so the install and upload traffic is not throttled. ### Do I need GitHub to deploy test procedures? URL: https://www.tofupilot.com/questions/do-i-need-github-to-deploy-test-procedures No. Git integration with GitHub or GitLab enables auto-deploy on every push, but you can also deploy from any machine without a connected repo using "tofupilot link" and "tofupilot deploy" from your local working tree. ### How do I import STDF files into TofuPilot? URL: https://www.tofupilot.com/questions/how-do-i-import-stdf-files-into-tofupilot You need an STDF v4 datalog (.stdf) and a procedure in the dashboard. STDF is structured binary and requires no column mapping. Drag the .stdf file into the dropzone (shows "Auto-detected") and click Import; TofuPilot detects STDF by its binary FAR record header, not the filename. A multi-part STDF file imports as one run per part, so a full wafer or lot lands as a batch of runs. Via API, call POST /v2/import with importer STDF (up to 100 files per call). ### Does TofuPilot integrate with my MES or ERP? URL: https://www.tofupilot.com/questions/does-tofupilot-integrate-with-my-mes-or-erp Yes, in both directions. Dashboard-side workflows push run results, serial numbers and batch data into your system, with a native Odoo integration, a generic HTTP action that calls any REST API, and custom integrations such as SAP for Enterprise customers. First Resonance and Epsilon3 integrations are in progress. Execution-side, a phase can query your MES before any test runs and stop the unit if it is refused, with a dedicated status so a blocked unit is never counted as a test failure. ### Is it safe to connect my factory stations to TofuPilot? URL: https://www.tofupilot.com/questions/is-it-safe-to-connect-my-factory-stations-to-tofupilot Yes, connecting your test stations to TofuPilot is safe. With our Pro plan, you can generate a secure API key for each station, with permissions restricted to specific actions such as uploading test results on specific test procedures. ### What if my stations can't connect to the internet? URL: https://www.tofupilot.com/questions/what-if-my-stations-cant-connect-to-the-internet If your stations can't connect publicly, you can either only allow access to TofuPilot URLs or collect test data on your side and upload it later. The original timestamps are preserved. ### What if my test infrastructure is fully offline? URL: https://www.tofupilot.com/questions/what-if-my-test-infrastructure-is-fully-offline We recommend our Enterprise plan for self-hosting TofuPilot within your infrastructure, keeping everything offline while maintaining all features. ### How secure is TofuPilot Cloud? URL: https://www.tofupilot.com/questions/how-secure-is-tofupilot-cloud All TofuPilot plans include secure cloud hosting managed by our team, and powered by AWS. Our team manages infrastructure with end-to-end communication encryption, strict internal role-based access, daily backups and regular audits to protect your sensitive test data. ### Can TofuPilot be self-hosted? URL: https://www.tofupilot.com/questions/can-tofupilot-be-self-hosted Yes, on the Enterprise plan. The dashboard runs on your own servers so no test data leaves your network, and we work with your IT team on the deployment. The self-hosted instance runs fully offline, including licence activation, and stations queue results locally when connectivity drops. ### Can I export my data from TofuPilot? URL: https://www.tofupilot.com/questions/can-i-export-my-data-from-tofupilot Yes, you can export data using our Python client or REST API or request a full extract from our team. ### What support does TofuPilot provide? URL: https://www.tofupilot.com/questions/what-support-does-tofupilot-provide Support levels differ by plan: Lab users can access support through Discord, Pro users via email, and Enterprise users with a dedicated contact in our team. ### Who is behind TofuPilot? URL: https://www.tofupilot.com/questions/who-is-behind-tofupilot TofuPilot is developed by TofuPilot SA, an award-winning Swiss startup founded by robotics test engineers. ### What is the TofuPilot product roadmap? URL: https://www.tofupilot.com/questions/what-is-the-tofupilot-product-roadmap Our roadmap is dedicated to streamlining your test development processes and improving analytics and insights. You can view our full roadmap to see what we're working on. ### When am I billed? URL: https://www.tofupilot.com/questions/when-am-i-billed Pro plans are billed monthly based on selected options and last month's usage. Enterprise plans offer monthly or annual billing to suit your needs. ### What payment methods do you accept? URL: https://www.tofupilot.com/questions/what-payment-methods-do-you-accept We accept payments from all major card providers for Pro plans, processed securely by our partner Stripe. Enterprise clients can request custom billing options. ### How do I back up a self-hosted TofuPilot instance? URL: https://www.tofupilot.com/questions/how-do-i-back-up-a-self-hosted-tofupilot-instance Your data lives in two Docker volumes: tofupilot-pg-data for the PostgreSQL database and tofupilot-seaweed-data for file storage. Back up both volumes on a regular schedule so a restore recovers your full instance. ### Can TofuPilot be self-hosted on my own servers? URL: https://www.tofupilot.com/questions/can-tofupilot-be-self-hosted-on-my-own-servers Yes, on the Enterprise plan. You run the dashboard, deployer, and PostgreSQL in your own infrastructure with Docker Compose, one organization per instance. Install with: curl -fsSL https://tofupilot.sh/deploy | bash. ### What are the server requirements for self-hosting TofuPilot? URL: https://www.tofupilot.com/questions/what-are-the-server-requirements-for-self-hosting-tofupilot Ubuntu 20.04+ or Debian on x86_64, 2+ CPU cores, 4 GB RAM, 40 GB+ disk, standard root Docker (not rootless), ports 80/443 open, and three subdomains for the dashboard, storage, and deployer. ### Does TofuPilot support SSO with Entra ID, Okta, or Google Workspace? URL: https://www.tofupilot.com/questions/does-tofupilot-support-sso-with-entra-id-okta-or-google-workspace Yes. Single sign-on via SAML 2.0 and OIDC works with Okta, Microsoft Entra, Azure AD, Auth0, OneLogin, and Google Workspace. SSO is available on Enterprise plans, and SCIM provisioning syncs organization members and roles from your directory. ### Does TofuPilot depend on the EU-US Data Privacy Framework? URL: https://www.tofupilot.com/questions/does-tofupilot-depend-on-the-eu-us-data-privacy-framework No. The EU-US Data Privacy Framework is required for US companies transferring EU personal data to the US. Since TofuPilot is Swiss-based with EU-hosted infrastructure, we do not rely on transfer frameworks that could be invalidated, as happened with Safe Harbor in 2015 and Privacy Shield in 2020. ### What user roles does TofuPilot have? URL: https://www.tofupilot.com/questions/what-user-roles-does-tofupilot-have Five roles: Owner, Admin, Developer, Viewer, and Operator. Teams additionally scope stations and data by department, supplier, or production line. ### How do I remove a team member in TofuPilot? URL: https://www.tofupilot.com/questions/how-do-i-remove-a-team-member-in-tofupilot Open Settings > Members. Ban blocks a user's sign-in without deleting their data and is reversible; Remove revokes access permanently. Both end access immediately. ### Does TofuPilot support SCIM provisioning? URL: https://www.tofupilot.com/questions/does-tofupilot-support-scim-provisioning Yes. Your identity provider pushes member changes to TofuPilot, so adding a user in Okta creates a TofuPilot account and removing them from the group revokes access. SCIM mirrors user creation, removal, attribute updates, suspension, and group changes. SCIM runs on top of SSO, so it requires SSO. Configure it under Settings > Organization > SCIM Provisioning to generate a bearer token and base URL, then map groups to roles in your IdP. The Owner role cannot be assigned via SCIM. ### Where is TofuPilot cloud data hosted? URL: https://www.tofupilot.com/questions/where-is-tofupilot-cloud-data-hosted Your core data (database and file storage) is hosted in Europe, operated by TofuPilot SA, a Swiss company not subject to the US CLOUD Act. Compute runs on a serverless architecture that routes each request to the nearest data center, so the platform stays fast for teams worldwide while data at rest remains in Europe. Customers needing strict EU-only data residency can get a managed dedicated instance hosted entirely in the EU, or self-host. ### How does TofuPilot secure my test data and access? URL: https://www.tofupilot.com/questions/how-does-tofupilot-secure-my-test-data-and-access Data is encrypted in transit and at rest. Access is controlled by role-based permissions and scoped API keys, and two-factor authentication supports passkeys (WebAuthn) and authenticator apps (TOTP). TofuPilot SA is Swiss-incorporated, so your data sits outside the reach of US surveillance law such as the CLOUD Act. Teams needing full control can self-host. ### Does TofuPilot support two-factor authentication (2FA)? URL: https://www.tofupilot.com/questions/does-tofupilot-support-two-factor-authentication-2fa Yes. TofuPilot supports two methods: passkeys (WebAuthn backed by biometrics, phone, or a hardware key like YubiKey) and authenticator apps (TOTP from 1Password, Google Authenticator, or Authy). Enable it under Settings > Authentication and save your one-time recovery codes; you can register multiple passkeys. Admins can require 2FA for all members under Settings > Organization > Security, and SSO members satisfy the requirement when their IdP enforces 2FA upstream. ### How do teams scope data in TofuPilot? URL: https://www.tofupilot.com/questions/how-do-teams-scope-data-in-tofupilot Teams scope which stations and runs a member sees. A station belongs to one team, but a member can belong to many. Owners, Admins, and Developers see every team's stations and runs. Viewers with no team see all data read-only; Viewers assigned to teams see only those. Operators see only stations in their assigned teams. A station with no team is visible to every Viewer and no Operator. Deleting a team keeps its stations and members, removing only the scoping link. ### Does TofuPilot work in air-gapped environments? URL: https://www.tofupilot.com/questions/does-tofupilot-work-in-air-gapped-environments Yes. The self-hosted deployment runs fully offline. License activation without internet works by copying a license token from TofuPilot Orbit and pasting it under Settings > Subscription on your instance.