Build a Test Data Strategy

A test data strategy decides what you capture, how you name it, and how you organize it before you write your first test. Getting this right early saves months of cleanup later. Getting it wrong means your analytics are noisy, your traceability has gaps, and your team wastes time decoding inconsistent data.

What Data to Capture

Every test run should include four categories of data:

Measurements with Limits

These are the quantitative values that determine pass/fail. Always define limits in code, not in post-processing. This ensures every run is evaluated against the same criteria.

Unit Identity

Every unit needs a unique serial number. If your product has sub-assemblies, track those serial numbers too. This is what lets you trace a field failure back through every test it ever went through.

Metadata

Context that doesn't have limits but matters for analysis: firmware version, hardware revision, operator ID, test station name. When you're investigating why yield dropped on Tuesday, metadata is how you find the cause.

Attachments

Log files, waveform captures, images from optical inspection. Not every run needs attachments, but when a failure investigation starts, you'll want them.

Naming Conventions

Consistent naming is the difference between data you can query and data you have to dig through manually.

Procedure Names

Use a clear hierarchy: {product}_{stage}_{test_type}. Keep names lowercase with underscores.

Don't embed dates or station IDs in procedure names. TofuPilot tracks those as separate fields.

Measurement Names

Use {component}_{parameter} format. Be specific enough that someone unfamiliar with the test can understand what was measured.

# Well-named measurements for a power supply test
import openhtf as htf
from openhtf.util import units
from tofupilot.openhtf import TofuPilot

@htf.measures(
    # Good: specific component, clear parameter
    htf.Measurement("rail_3v3_voltage")
        .in_range(minimum=3.25, maximum=3.35)
        .with_units(units.VOLT),
    htf.Measurement("rail_3v3_ripple")
        .in_range(maximum=30)
        .with_units(units.MILLIVOLT),
    htf.Measurement("rail_5v_voltage")
        .in_range(minimum=4.9, maximum=5.1)
        .with_units(units.VOLT),
    htf.Measurement("rail_5v_load_regulation_pct")
        .in_range(maximum=2.0),
    htf.Measurement("input_current_idle")
        .in_range(maximum=0.050)
        .with_units(units.AMPERE),
    htf.Measurement("thermal_shutdown_temp")
        .in_range(minimum=145, maximum=155)
        .with_units(units.DEGREE_CELSIUS),
)
def power_supply_validation(test):
    test.measurements.rail_3v3_voltage = 3.301
    test.measurements.rail_3v3_ripple = 18.4
    test.measurements.rail_5v_voltage = 5.03
    test.measurements.rail_5v_load_regulation_pct = 1.2
    test.measurements.input_current_idle = 0.0321
    test.measurements.thermal_shutdown_temp = 150.2

def main():
    test = htf.Test(power_supply_validation)
    with TofuPilot(test):
        test.execute(test_start=lambda: "PSU-2024-0891")

if __name__ == "__main__":
    main()

Avoid these naming mistakes:

Serial Number Format

Pick a format and enforce it. Common patterns:

Organizing Procedures by Production Phase

Structure your procedures to match your actual production flow. Each phase of testing should be a separate procedure in TofuPilot.

This structure gives you per-phase FPY, lets you compare yield between EVT and PVT, and makes it obvious which test caught a failure.

Adding Metadata with OpenHTF

Use TofuPilot parameters to attach context that isn't a measurement:

# Attaching sub-unit serial numbers for BOM traceability
import openhtf as htf
from tofupilot.openhtf import TofuPilot

@htf.measures(
    htf.Measurement("boot_time_ms").in_range(maximum=500),
    htf.Measurement("self_test_result").equals("PASS"),
)
def functional_check(test):
    test.measurements.boot_time_ms = 230
    test.measurements.self_test_result = "PASS"

def main():
    test = htf.Test(functional_check)
    with TofuPilot(
        test,
        procedure_id="sensor_v2_pvt_functional",
        sub_units=[
            {"serial_number": "WIFI-MOD-2024-0331"},
            {"serial_number": "BT-MOD-2024-0887"},
        ],
    ):
        test.execute(test_start=lambda: "SENSOR-2024-2210")

if __name__ == "__main__":
    main()

Sub-unit serial numbers let you trace which specific components went into each assembly. When a component lot has issues, you can query TofuPilot to find every finished unit that contains affected parts.

Data Retention Checklist

Before you start collecting data, answer these questions:

1. What compliance standards apply? ISO 13485 (medical), AS9100 (aerospace), and IATF 16949 (automotive) all have specific data retention requirements.
2. How long do you need to keep data? Product lifetime plus warranty period is a common baseline. Medical devices often require 15+ years.
3. What data needs to be immutable? Test results used for regulatory compliance should never be editable after the fact.
4. Who needs access? Define roles early. Test engineers, quality managers, and customers may need different views of the same data.

TofuPilot stores all test data with full audit history. Every run is timestamped and linked to its procedure, station, and operator. You don't need to build your own retention system.