A test data strategy decides what you capture, how you name it, and how you organize it before you write your first test. Getting this right early saves months of cleanup later. Getting it wrong means your analytics are noisy, your traceability has gaps, and your team wastes time decoding inconsistent data.
What Data to Capture
Every test run should include four categories of data:
Measurements with Limits
These are the quantitative values that determine pass/fail. Always define limits in code, not in post-processing. This ensures every run is evaluated against the same criteria, and it means the limit that applied travels with the result.
| Data Type | Example | Why It Matters |
|---|---|---|
| Parametric measurements | Voltage, current, resistance | Enables Cpk analysis and trend detection |
| Functional checks | Communication response, boot time | Validates system-level behavior |
| Environmental readings | Temperature, humidity during test | Explains measurement variation |
Record the measured value, not just the verdict. A boolean tells you a unit failed; the value tells you it measured 3.19V against a 3.20V limit and had been drifting for three weeks. This is the single field that is impossible to reconstruct later.
Unit Identity
Every unit needs a unique serial number. If your product has sub-assemblies, track those serial numbers too. This is what lets you trace a field failure back through every test it ever went through.
Metadata
Context that doesn't have limits but matters for analysis: firmware version, hardware revision, operator, test station. When you're investigating why yield dropped on Tuesday, metadata is how you find the cause.
Attachments
Log files, waveform captures, images from optical inspection. Not every run needs attachments, but when a failure investigation starts, you'll want them. Attaching on failures rather than on every unit keeps this from outgrowing its usefulness.
Naming Conventions
Consistent naming is the difference between data you can query and data you have to dig through manually.
Procedure Names
Use a clear hierarchy: {product}_{stage}_{test_type}. Keep names lowercase with underscores.
| Pattern | Example |
|---|---|
{product}_{stage}_{test} | sensor_v2_evt_functional |
{product}_{stage}_{test} | motor_ctrl_pvt_burn_in |
{product}_{stage}_{test} | power_supply_dvt_thermal |
Don't embed dates or station IDs in procedure names. Those are tracked as separate fields.
Measurement Names
Use {component}_{parameter} format. Be specific enough that someone unfamiliar with the test can understand what was measured.
test_naming_example.py47 lines
# Well-named measurements for a power supply testimport openhtf as htffrom openhtf.util import unitsfrom tofupilot.openhtf import upload@htf.measures( # Good: specific component, clear parameter htf.Measurement("rail_3v3_voltage") .in_range(minimum=3.25, maximum=3.35) .with_units(units.VOLT), htf.Measurement("rail_3v3_ripple") .in_range(maximum=30) .with_units(units.MILLIVOLT), htf.Measurement("rail_5v_voltage") .in_range(minimum=4.9, maximum=5.1) .with_units(units.VOLT), htf.Measurement("rail_5v_load_regulation_pct") .in_range(maximum=2.0), htf.Measurement("input_current_idle") .in_range(maximum=0.050) .with_units(units.AMPERE), htf.Measurement("thermal_shutdown_temp") .in_range(minimum=145, maximum=155) .with_units(units.DEGREE_CELSIUS),)def power_supply_validation(test): test.measurements.rail_3v3_voltage = 3.301 test.measurements.rail_3v3_ripple = 18.4 test.measurements.rail_5v_voltage = 5.03 test.measurements.rail_5v_load_regulation_pct = 1.2 test.measurements.input_current_idle = 0.0321 test.measurements.thermal_shutdown_temp = 150.2def main(): test = htf.Test( power_supply_validation, procedure_id="xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx", # procedure UUID from the dashboard part_number="PSU-100", ) test.add_output_callbacks(upload()) test.execute(lambda: "PSU-2024-0891")if __name__ == "__main__": main()Install it with pip install "tofupilot[openhtf]". The upload() callback reads TOFUPILOT_API_KEY from the environment.
Measurement names are effectively permanent. Renaming one splits its history into two series, so the trend you wanted to look at stops at the rename. Settle the convention before the first production run.
Avoid these naming mistakes:
| Bad Name | Problem | Better Name |
|---|---|---|
test1 | Meaningless | rail_3v3_voltage |
voltage | Which voltage? | rail_5v_voltage |
V_out_3.3V_meas | Inconsistent casing and format | rail_3v3_output_voltage |
temperature | Which temperature, what unit? | thermal_shutdown_temp |
Serial Number Format
Pick a format and enforce it. Common patterns:
| Format | Example | Use Case |
|---|---|---|
{PREFIX}-{YEAR}-{SEQ} | PCB-2024-00042 | General manufacturing |
{PRODUCT}-{LOT}-{SEQ} | SNS-L0287-015 | Lot-based production |
{SITE}-{LINE}-{DATE}-{SEQ} | SH-A3-240315-0001 | Multi-site tracking |
Validate the format at the scan prompt. A typo caught at the fixture costs seconds; the same typo found six months later in a traceability record is usually unrecoverable.
Organizing Procedures by Production Phase
Structure your procedures to match your actual production flow. Each phase of testing should be a separate procedure.
EVT (Engineering Validation)
├── sensor_v2_evt_power_on
├── sensor_v2_evt_functional
├── sensor_v2_evt_environmental
└── sensor_v2_evt_reliability
DVT (Design Validation)
├── sensor_v2_dvt_incoming_inspection
├── sensor_v2_dvt_calibration
├── sensor_v2_dvt_functional
└── sensor_v2_dvt_burn_in
PVT (Production Validation)
├── sensor_v2_pvt_smt_inspection
├── sensor_v2_pvt_ict
├── sensor_v2_pvt_functional
└── sensor_v2_pvt_final_qc
This structure gives you per-phase first pass yield, lets you compare yield between EVT and PVT, and makes it obvious which test caught a failure.
Recording Sub-Assembly Links
sub_units is a field on htf.Test(...), alongside procedure_id and part_number. It records which components went into the unit being tested.
test_with_metadata.py30 lines
# Attaching sub-unit serial numbers for BOM traceabilityimport openhtf as htffrom tofupilot.openhtf import upload@htf.measures( htf.Measurement("boot_time_ms").in_range(maximum=500), htf.Measurement("self_test_result").equals("PASS"),)def functional_check(test): test.measurements.boot_time_ms = 230 test.measurements.self_test_result = "PASS"def main(): test = htf.Test( functional_check, procedure_id="xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx", # UUID for sensor_v2_pvt_functional part_number="SENSOR-V2", sub_units=[ {"serial_number": "WIFI-MOD-2024-0331"}, {"serial_number": "BT-MOD-2024-0887"}, ], ) test.add_output_callbacks(upload()) test.execute(lambda: "SENSOR-2024-2210")if __name__ == "__main__": main()Note that procedure_id is the dashboard UUID, not the procedure name. The naming convention above is for humans reading the dashboard; the code references the UUID. A legacy external identifier also works, but only if one was explicitly set on the procedure.
Sub-unit serial numbers let you trace which specific components went into each assembly. When a component lot has issues, you can find every finished unit that contains affected parts instead of quarantining a date range. This is also the hardest field to add retroactively, so model it from the first build even if you do not query it yet.
Data Retention Checklist
Before you start collecting data, answer these questions:
- What compliance standards apply? ISO 13485 (medical), AS9100 (aerospace), and IATF 16949 (automotive) all have specific data retention requirements.
- How long do you need to keep data? Product lifetime plus warranty period is a common baseline. Medical devices often require 15+ years.
- What data needs to be immutable? Test results used for regulatory compliance should never be editable after the fact.
- Who needs access? Define roles early. Test engineers, quality managers, and customers may need different views of the same data.
Add a fifth: how do you get the data out? Retention obligations outlive tooling choices, so keep an export path that does not depend on a single vendor remaining in business.
Test data is stored with full audit history. Every run is timestamped and linked to its procedure, station, and operator, so you do not need to build your own retention system.
