Fault injection campaign sizer

How it works

A fault injection campaign shows what share of dangerous faults a safety mechanism catches, which feeds the diagnostic coverage in the hardware metrics. Injecting every fault is rarely affordable, so the campaign draws a random sample and states a margin of error. This tool sizes that sample, prices it in compute hours and calendar days, and shows the interval you can quote afterwards.

1. Build the fault list. Each fault model contributes sites. A stuck-at fault can sit on every net at value 0 or 1. A transient bit flip can hit every flip-flop or storage bit at any of the injection times you choose across the workload. Faults proved safe or equivalent by static analysis come off the list. Other fault models (bridging, delay, multi-bit) are added as entered counts.N = (nets + nets + flip-flops x injection times) x (1 − pruned share) + other faults
2. Choose the confidence factor. The confidence level fixes a cut-off t on the standard normal curve, taken two-sided. 90% gives 1.645, 95% gives 1.960 and 99% gives 2.576.t = z(1 − (1 − confidence) / 2)
3. Size the sample. Leveugle and co-workers size a random sample of a finite fault list. e is the margin of error you accept on the measured proportion and p is the proportion you expect. The value p = 0.5 is the worst case and needs the most runs. A prior estimate of coverage, say 0.99, needs far fewer but is only safe if the estimate is good. For a very large list the result tends to t^2 p (1 − p) / e^2, the textbook sample size for a proportion.n = N / (1 + e^2 (N − 1) / (t^2 p (1 − p)))
4. Allow for masked faults. Coverage is measured over faults that could cause a dangerous effect. Faults that turn out safe or masked do not count, so they must be injected too. If a share s of injections is safe, the runs needed rise.runs = n / (1 − s)
5. Compute hours and calendar days. The run time is either entered in minutes or worked out from the cycles per run and the simulator speed. Early termination, where a run stops once the checker fires or the fault is masked, shortens the average run; a fixed overhead per run covers checkpoint restore and result comparison. Machines or licences work in parallel, and the days add the set-up time before the first injection.run minutes = base minutes x simulated share + overhead; compute hours = runs x run minutes / 60; calendar days = set-up days + compute hours / (machines x hours per day)
6. Read the margin a budget buys. If the schedule fixes the number of runs, solve the sample size formula for e. At the worst case p = 0.5 this is the largest margin you will see.e = t x sqrt(p (1 − p) (N − n) / (n (N − 1)))
7. Quote the coverage. After the campaign, the measured coverage is the detected count over the counted injections. The sizing formula gives a normal interval around it; the Clopper-Pearson interval is the exact binomial one and is a little wider. Both are shown for the coverage you expect.coverage = detected / n; normal interval = coverage ± t x sqrt(coverage (1 − coverage) / n x (N − n) / (N − 1))
8. The exact view: prove a target. To claim that coverage is at least a target, the one-sided lower Clopper-Pearson limit must reach it. For an expected coverage c, the tool finds the fewest injections n for which that limit, with n x c detections, is at or above the target. When no escape is expected, the answer is the success-run count.lower limit = Beta quantile (alpha; k, n − k + 1) with k = n x c; no escapes: n = ln(alpha) / ln(target)

Sources

The methods and default figures rest on these public sources. The wording here is our own; read the source for its full text.

• Statistical fault injection: Quantified error and confidence (opens in a new tab), R. Leveugle, A. Calvez, P. Maistri and P. Vanhauwaert, Design, Automation and Test in Europe (DATE), IEEE, pages 502 to 506, April 2009. The finite-population sample size for a fault list, with the margin of error, confidence and assumed proportion used here.
• ISO 26262-11:2018, Road vehicles, Functional safety, Part 11: Guidelines on application of ISO 26262 to semiconductors, International Organization for Standardization, December 2018. Clause 4.8 Fault injection (characteristics or variables of fault injection, fault injection results), clause 5.1.2 and 5.1.3 fault models of digital components and memories, and clause 5.1.10 verification using fault injection simulation. Cited by name and clause only.
• ISO 26262-5:2018, Road vehicles, Functional safety, Part 5: Product development at the hardware level, International Organization for Standardization, December 2018. Clause 8 evaluation of the hardware architectural metrics (single-point and latent fault metrics) and Annex D evaluation of diagnostic coverage. Cited by name and clause only.
• The use of confidence or fiducial limits illustrated in the case of the binomial (opens in a new tab), C. J. Clopper and E. S. Pearson, Biometrika, volume 26, issue 4, pages 404 to 413 (Oxford University Press), 1934. The exact binomial limits used in the interval and in the exact view.