Pareto Analysis for Test Failures

The Pareto principle says 80% of your test failures come from 20% of your failure modes. If you fix the right two or three issues, most of your yield problems disappear. But you need structured failure data to find those issues, and that starts with how you write your tests.

What Pareto Analysis Tells You

A Pareto chart ranks failure modes by frequency, highest to lowest, with a cumulative line showing how much of total failures each mode accounts for. It answers one question: where should you spend your time?

Without a Pareto chart, teams chase whatever failure they saw most recently. With one, you can see that "clock frequency out of range" causes 40% of all failures while "communication timeout" causes 3%. Fixing the clock issue first gives you 13x more impact.

Writing Tests with Clear Failure Modes

Pareto analysis only works if your test phases map to distinct, diagnosable failure modes. A single phase that checks 15 things produces one failure label when any of them breaks. You can't tell which sub-check caused it.

Split your test into phases that each cover one functional area.

import openhtf as htf
from openhtf.util import units
from tofupilot.openhtf import TofuPilot

@htf.measures(
    htf.Measurement("input_voltage")
    .with_units(units.VOLT)
    .in_range(minimum=4.75, maximum=5.25),
    htf.Measurement("inrush_current")
    .in_range(maximum=500),
)
def test_power_input(test):
    test.measurements.input_voltage = 5.02
    test.measurements.inrush_current = 320

@htf.measures(
    htf.Measurement("rail_3v3_voltage")
    .with_units(units.VOLT)
    .in_range(minimum=3.135, maximum=3.465),
    htf.Measurement("rail_3v3_ripple")
    .in_range(maximum=50),
)
def test_3v3_regulator(test):
    test.measurements.rail_3v3_voltage = 3.28
    test.measurements.rail_3v3_ripple = 22

@htf.measures(
    htf.Measurement("clock_frequency")
    .in_range(minimum=7.99, maximum=8.01),
    htf.Measurement("clock_jitter")
    .in_range(maximum=100),
)
def test_clock(test):
    test.measurements.clock_frequency = 8.003
    test.measurements.clock_jitter = 45

@htf.measures(
    htf.Measurement("spi_loopback_pass"),
    htf.Measurement("spi_throughput")
    .in_range(minimum=900),
)
def test_spi_interface(test):
    test.measurements.spi_loopback_pass = True
    test.measurements.spi_throughput = 980

@htf.measures(
    htf.Measurement("adc_offset_error")
    .in_range(minimum=-2, maximum=2),
    htf.Measurement("adc_gain_error_pct")
    .in_range(minimum=-0.5, maximum=0.5),
    htf.Measurement("adc_snr")
    .in_range(minimum=60),
)
def test_adc_accuracy(test):
    test.measurements.adc_offset_error = 0.8
    test.measurements.adc_gain_error_pct = -0.12
    test.measurements.adc_snr = 67.3

def main():
    test = htf.Test(
        test_power_input,
        test_3v3_regulator,
        test_clock,
        test_spi_interface,
        test_adc_accuracy,
    )
    with TofuPilot(test):
        test.execute(test_start=lambda: "SN-2026-01580")

if __name__ == "__main__":
    main()

Each phase covers one functional block: power input, voltage regulation, clock, SPI communication, and ADC accuracy. When a unit fails, the failure is tagged to a specific phase and measurement. TofuPilot uses this structure to build the Pareto chart.

Reading TofuPilot's Failure Pareto Chart

TofuPilot's analytics dashboard includes a failure Pareto chart that updates in real time as test data flows in. Here's how to use it.

Select your scope. Filter by product, station, or time range. A Pareto chart for all products combined isn't useful because different products have different failure profiles.

Read the bars. Each bar represents a failure mode (a test phase or specific measurement that failed). The tallest bar on the left is your biggest problem.

Follow the cumulative line. The line shows what percentage of total failures you've covered as you move left to right. If the first two bars reach 70% on the cumulative line, fixing those two issues eliminates 70% of your failures.

Compare over time. After fixing the top failure mode, check the Pareto chart again. The distribution will shift. What was previously the second-biggest issue might now be the top one, or a previously hidden issue might surface.

Turning Pareto Data into Action

A Pareto chart gives you priorities. Turning those priorities into fixes requires a systematic approach.

For the Top Failure Mode

1. Pull up the measurement histogram in TofuPilot for the failing measurement. Check whether failures cluster at one limit or spread across the range.
2. Check the Cpk. If Cpk is below 1.0, your process isn't capable of meeting the test limits consistently. You either need to improve the process or review whether the limits are too tight.
3. Look at the unit history for several failed serial numbers. Do they fail once and pass on retest (fixture issue), or do they fail consistently (real defect)?
4. Check for station correlation. If failures concentrate on one station, the problem is likely the fixture or test environment, not the product.

For Recurring Low-Level Failures

Some failure modes sit at 2-5% individually but collectively account for 20-30% of failures. These are often fixture wear, environmental drift, or operator handling issues. Address them as a group by improving fixture maintenance schedules and adding fixture validation phases to your tests.

What Good Looks Like

A healthy Pareto chart has no single failure mode dominating above 15-20%. The bars taper gradually from left to right, indicating no single systemic issue.

If your chart shows one bar towering over the rest, you have a known problem that hasn't been addressed. If the chart is nearly flat with many small bars, your failures are distributed and likely random, which means your process is well-controlled and further improvement requires tightening tolerances or upgrading components.

Review the Pareto chart weekly during DVT and PVT. In production, a monthly review catches new trends before they become costly.