The situation is familiar. Pre-production runs were fine. Same parts, same gerbers, same test. Then production starts and a chunk of boards fail, often at one specific step, and nothing in the design changed.
This guide is a method for finding the cause. It assumes you have test results recorded. If you only have pass and fail counts, skip to the last section, because that is the actual problem and it is worth fixing before the next time.
The queries below assume measurements are recorded per step, with each step belonging to a run: run → step → measurement. Adjust the joins if your schema attaches measurements directly to runs.
Start with the measured values, not the pass rate
The pass rate tells you units failed. It does not tell you why, and it hides the most useful signal.
A measurement that is drifting toward its limit will pass for weeks before it starts failing. By the time the yield number moves, the cause has been present for a long time. If you plot the measured values rather than the outcomes, the drift is visible before it costs you anything.
SELECT date_trunc('day', r.started_at) AS day, avg(m.value) AS mean, stddev(m.value) AS sigma, min(m.value) AS min_value, max(m.value) AS max_value, count(*) AS nFROM measurement mJOIN step s ON s.id = m.step_idJOIN run r ON r.id = s.run_idWHERE m.name = 'rail_3v3' AND r.procedure_id = 'FCT-001'GROUP BY dayORDER BY day;Look at the mean and the spread separately. They fail differently:
- Mean moves, spread stays tight. Something shifted. A new component lot, a reflow profile change, a recalibrated instrument, a different power supply on the bench.
- Mean stays, spread widens. Something became inconsistent. A worn fixture, an intermittent contact, temperature variation across a shift.
- Both move. Usually two causes, or one cause with a knock-on effect. Split the data before going further.
Split the data before forming a theory
Before you theorise, cut the failures by every dimension you record. The point is to find a dimension where the failure rate is not uniform.
| Split by | What a difference means |
|---|---|
| Test station | Fixture wear, a miscalibrated instrument, a bad cable on one station |
| Time of day or shift | Temperature, mains variation, a specific operator, warm-up effects |
| Component lot or date code | Incoming part variation, the most common cause of a sudden step change |
| PCB panel or fab lot | Board-level process issue |
| Operator | Fixture seating technique, a step being skipped |
| Test step | Narrows which physical subsystem is involved |
| Firmware version | A calibration constant or threshold changed in software |
SELECT r.station_id, count(*) AS runs, count(*) FILTER (WHERE NOT r.passed) AS failures, round(100.0 * count(*) FILTER (WHERE NOT r.passed) / count(*), 2) AS fail_pctFROM run rWHERE r.procedure_id = 'FCT-001' AND r.started_at > now() - interval '14 days'GROUP BY r.station_idORDER BY fail_pct DESC;If one station is at 12% and the others are at 2%, you have your answer and it is not a design problem. This single query resolves a surprising share of yield investigations.
Confirm the failure is real before chasing it
Before investigating a process, rule out the test itself. A test that fails good boards is more common than people expect, and it wastes weeks.
Check in this order:
- Retest a failing unit without touching it. If it passes, you have an intermittent contact or a marginal limit, not a product defect.
- Retest on a different station. If it passes, the first station is the problem.
- Swap the suspect part between a failing board and a passing board. If the failure follows the part, it is a component issue. If it stays with the board, it is the board or the process.
- Measure by hand with a separate instrument. If the manual measurement disagrees with the fixture, the fixture or its calibration is suspect.
Step 3 is the strongest diagnostic available and it costs one rework cycle. It cleanly separates component problems from assembly problems.
Check whether the limits are the problem
If the measured values have not moved but the failure rate has, look at the limits rather than the process.
Limits often get tightened after a field failure, or copied from a datasheet without accounting for measurement uncertainty. A limit that sits close to the natural spread of your process will reject good units at a steady rate.
The rule of thumb is that your process spread should fit comfortably inside the limits. If the distance between your limit and your mean is less than about four standard deviations, expect rejects from normal variation alone. That ratio is what process capability, Cp and Cpk, measures.
SELECT m.name, avg(m.value) AS mean, stddev(m.value) AS sigma, min(m.lower_limit) AS lsl, min(m.upper_limit) AS usl, least( (min(m.upper_limit) - avg(m.value)) / (3 * stddev(m.value)), (avg(m.value) - min(m.lower_limit)) / (3 * stddev(m.value)) ) AS cpkFROM measurement mJOIN step s ON s.id = m.step_idJOIN run r ON r.id = s.run_idWHERE r.procedure_id = 'FCT-001' AND r.started_at > now() - interval '30 days'GROUP BY m.nameHAVING stddev(m.value) > 0ORDER BY cpk ASC;Sort ascending and the measurements most likely to cause rejects appear first. Anything below 1.33 is worth attention; below 1.0 you are rejecting units through normal variation.
Two caveats before acting on the number. Cpk assumes the measurement is roughly normally distributed and the process is stable over the window you queried, so a value computed across a drift is meaningless. And a one-sided measurement, where only an upper or lower limit exists, makes one half of that least() null. Handle that case separately rather than reading the result as a low score.
Which test step is actually costing you
When several steps fail, fix the one that costs the most units rather than the one that is most interesting.
SELECT s.name AS step, count(*) AS failures, round(100.0 * count(*) / sum(count(*)) OVER (), 1) AS pct_of_failuresFROM step sJOIN run r ON r.id = s.run_idWHERE r.procedure_id = 'FCT-001' AND NOT s.passed AND r.started_at > now() - interval '30 days'GROUP BY s.nameORDER BY failures DESC;Failure distributions are usually heavily skewed. Two or three steps account for most of the loss, and the rest is noise.
Watch out for retests hiding the problem
If your yield number counts any passing run rather than the first run per unit, retests will mask a worsening process. A unit that fails twice and passes on the third attempt looks identical to a unit that passed immediately.
First pass yield means the first run per unit:
SELECT date_trunc('day', first_run.started_at) AS day, count(*) FILTER (WHERE first_run.passed) * 100.0 / count(*) AS fpy_percent, count(*) AS unitsFROM ( SELECT DISTINCT ON (unit_id) unit_id, started_at, passed FROM run WHERE procedure_id = 'FCT-001' ORDER BY unit_id, started_at ASC) AS first_runGROUP BY dayORDER BY day DESC;The DISTINCT ON with an ascending sort is what makes this the first run rather than any run. Getting this wrong is the most common reporting bug in home-built systems, and it always makes the number look better than reality.
A widening gap between first pass yield and final yield is itself a signal: the process is degrading and retests are absorbing it.
When you do not have the data
If the investigation above is impossible because you only recorded pass and fail, that is the finding. The fix is not complicated but it has to happen before the next incident:
Store the measured value, not just the outcome. A boolean tells you a unit failed. The value tells you it was drifting for three weeks beforehand. This single change makes most of the queries above possible.
Record which station, which operator, which firmware version. These are the dimensions you will want to split by, and they cost nothing to capture at test time.
Version the test procedure. When limits change, you need to know which version produced a given result, otherwise old data becomes uninterpretable.
Keep serial numbers and sub-assembly links. When a failure correlates with a component lot, you need to trace which units contain which parts.
Set an alert on the trend, not the threshold. By the time yield crosses a threshold the cause is weeks old. Alerting on drift catches it while units are still passing.
TofuPilot records all of this by default from a Python test script, and computes yield, capability, control charts and failure Pareto without the queries above. But the queries are here because the method matters more than the tool, and they work against any schema that stores the measured value.