Side-channel leakage assessment
Run a fixed versus random Welch t-test on side-channel traces or summary statistics, flag leaky points against the threshold, and size the traces needed.
Safety and securityFree to useHow it worksSourcesWhere it helps
Fills the inputs below with typical figures. Replace any of them with your own.
- First-order test
- Leak at 2 points
- Largest |t| 10.59 at point 15
- First-order: df and p at the largest
- 973.5 df, p 7.13e-25
- Expected false alarms for a leak-free device: 0.00027
- Traces
- 500 fixed, 500 random
- 40 sample points
- Traces to see an effect of 0.1
- 9,320 per group
- 18,640 in all, at power 0.99 and threshold 4.5. With your counts the smallest effect seen is 0.432
- Chance of detecting that effect with your traces
- 0.2%
- Expected t 1.58
Sign in to save your inputs and come back to them later.
- t value (solid line, circles)
- Threshold +4.5 (dashed line, squares)
- Threshold −4.5 (dotted line, triangles)
Show the data as a table
| Sample point | t value | Threshold +4.5 | Threshold −4.5 |
|---|---|---|---|
| 1 | 0.858 | 4.5 | -4.5 |
| 2 | -1.28 | 4.5 | -4.5 |
| 3 | -0.851 | 4.5 | -4.5 |
| 4 | -0.0358 | 4.5 | -4.5 |
| 5 | -0.111 | 4.5 | -4.5 |
| 6 | 0.867 | 4.5 | -4.5 |
| 7 | -0.735 | 4.5 | -4.5 |
| 8 | -0.658 | 4.5 | -4.5 |
| 9 | 0.598 | 4.5 | -4.5 |
| 10 | 0.435 | 4.5 | -4.5 |
| 11 | -0.238 | 4.5 | -4.5 |
| 12 | -0.264 | 4.5 | -4.5 |
| 13 | 0.569 | 4.5 | -4.5 |
| 14 | 5.13 | 4.5 | -4.5 |
| 15 | 10.6 | 4.5 | -4.5 |
| 16 | 3.22 | 4.5 | -4.5 |
| 17 | 0.439 | 4.5 | -4.5 |
| 18 | 1.58 | 4.5 | -4.5 |
| 19 | 0.746 | 4.5 | -4.5 |
| 20 | -2.32 | 4.5 | -4.5 |
| 21 | -0.596 | 4.5 | -4.5 |
| 22 | 1.81 | 4.5 | -4.5 |
| 23 | -0.475 | 4.5 | -4.5 |
| 24 | 1.73 | 4.5 | -4.5 |
| 25 | -1.7 | 4.5 | -4.5 |
| 26 | 0.889 | 4.5 | -4.5 |
| 27 | 3.38 | 4.5 | -4.5 |
| 28 | 0.469 | 4.5 | -4.5 |
| 29 | 1.94 | 4.5 | -4.5 |
| 30 | 0.601 | 4.5 | -4.5 |
| 31 | -0.0788 | 4.5 | -4.5 |
| 32 | -1.01 | 4.5 | -4.5 |
| 33 | -0.526 | 4.5 | -4.5 |
| 34 | 0.885 | 4.5 | -4.5 |
| 35 | -1.01 | 4.5 | -4.5 |
| 36 | -0.472 | 4.5 | -4.5 |
| 37 | -0.28 | 4.5 | -4.5 |
| 38 | -0.267 | 4.5 | -4.5 |
| 39 | -1.21 | 4.5 | -4.5 |
| 40 | -1.14 | 4.5 | -4.5 |
| Order | Point | t | t in each set | Degrees of freedom | p-value |
|---|---|---|---|---|---|
| First | 15 | 10.59 | one set | 973.5 | 7.13e-25 |
| First | 14 | 5.13 | one set | 997.4 | 3.53e-7 |
- Power 0.5 (solid line, circles)
- Power 0.9 (dashed line, squares)
- Power 0.99 (dotted line, triangles)
Show the data as a table
| Standardised effect | Power 0.5 | Power 0.9 | Power 0.99 |
|---|---|---|---|
| 0.05 | 16,200 | 26,742 | 37,280 |
| 0.1 | 4,050 | 6,686 | 9,320 |
| 0.15 | 1,800 | 2,972 | 4,143 |
| 0.2 | 1,013 | 1,672 | 2,330 |
| 0.3 | 450 | 743 | 1,036 |
| 0.4 | 254 | 418 | 583 |
| 0.5 | 162 | 268 | 373 |
| 0.7 | 83 | 137 | 191 |
| 1 | 41 | 67 | 94 |
- Expected t (solid line, circles)
- Threshold 4.5 (dashed line, squares)
Show the data as a table
| Traces per group | Expected t | Threshold 4.5 |
|---|---|---|
| 0 | 0 | 4.5 |
| 1,560 | 2.79 | 4.5 |
| 3,120 | 3.95 | 4.5 |
| 4,680 | 4.84 | 4.5 |
| 6,240 | 5.59 | 4.5 |
| 7,800 | 6.24 | 4.5 |
| 9,360 | 6.84 | 4.5 |
| 10,920 | 7.39 | 4.5 |
| 12,480 | 7.9 | 4.5 |
| 14,040 | 8.38 | 4.5 |
| 15,600 | 8.83 | 4.5 |
| 17,160 | 9.26 | 4.5 |
| 18,720 | 9.67 | 4.5 |
| Standardised effect | Power 0.5 | Power 0.9 | Power 0.99 |
|---|---|---|---|
| 0.05 | 16,200 | 26,742 | 37,280 |
| 0.1 | 4,050 | 6,686 | 9,320 |
| 0.15 | 1,800 | 2,972 | 4,143 |
| 0.2 | 1,013 | 1,672 | 2,330 |
| 0.3 | 450 | 743 | 1,036 |
| 0.4 | 254 | 418 | 583 |
| 0.5 | 162 | 268 | 373 |
| 0.7 | 83 | 137 | 191 |
| 1 | 41 | 67 | 94 |
How it works
Test vector leakage assessment (TVLA) asks whether a device's power or electromagnetic trace depends on the data it handles, without attacking a key. It compares traces taken with one fixed input against traces taken with random inputs, sample point by sample point, with Welch's t-test. A |t| above 4.5 at any point is taken as evidence of leakage. This tool runs the test from summary statistics or from short pasted traces, adds the second-order test for masked designs, and tells you how many traces you need.
- Keep running moments, one pass. For each sample point the tool keeps the count, the mean and the central sums M2, M3 and M4 and updates them trace by trace, so no trace is stored twice and the numbers stay stable for very large campaigns. This is Welford's update, extended to the fourth moment.
delta = x − mean; mean = mean + delta / n; M2 = M2 + delta x (x − mean new); sample variance = M2 / (n − 1) - Welch's t-test per point. The fixed and random groups are compared at each point without assuming equal variances.
t = (mean fixed − mean random) / sqrt(var fixed / n fixed + var random / n random) - Degrees of freedom and p-value. The Welch-Satterthwaite formula gives the degrees of freedom. The two-sided p-value comes from the Student t distribution through the regularised incomplete beta function, which stays accurate far into the tail.
df = (a + b)^2 / (a^2 / (n fixed − 1) + b^2 / (n random − 1)), where a = var fixed / n fixed and b = var random / n random; p = I(x; df / 2, 1/2) with x = df / (df + t^2) - Second-order test. A well-masked design leaks nothing in the mean, but the spread of the trace can still depend on the secret. The second-order test centres each group's samples on its own mean and squares them, then applies the same Welch test to the squares. From the running sums, the mean of the squares is M2 / n and their variance is (M4 − M2^2 / n) / (n − 1).
y = (x − mean)^2; mean of y = M2 / n; var of y = (M4 − M2^2 / n) / (n − 1) - Decide. A point leaks when |t| is above the threshold, 4.5 by default, which corresponds to a two-sided p-value of about 7 in a million for each point. With two independent sets, a point counts only if both sets exceed the threshold with the same sign, which removes most chance alarms. The chance of at least one false alarm grows with the number of points, so the tool also gives the expected number of false alarms for an ideal device.
false alarms expected = points x 2 x (1 − Phi(threshold)); two sets: points x (2 x (1 − Phi(threshold)))^2 / 2 - How many traces are needed. For a standardised effect d (the difference of the two means divided by the standard deviation) and equal groups, the average t grows with the square root of the group size. The traces needed per group to reach the threshold with a chosen power use the normal approximation. The smallest effect found with the traces you have follows from the same relation.
traces per group = 2 x (threshold + z power)^2 / d^2; expected t = d / sqrt(1 / n fixed + 1 / n random); smallest effect = (threshold + z power) x sqrt(1 / n fixed + 1 / n random)
Assumptions
- The fixed and random groups are acquired in an interleaved order, with the same set-up, so drift in the environment affects both groups alike.
- At each sample point the readings in a group are independent and roughly normal, or the groups are large enough for the central limit theorem to hold. The second-order test needs far more traces than the first-order test for the same effect, because it works on squares.
- Example traces are seeded and synthetic: a baseline, leakage proportional to Hamming weights in narrow windows, and Gaussian noise. They teach the method; they are not measurements of any device.
- The sizing treats the effect as the standardised difference of means at one point, with equal group sizes, and uses the normal approximation to the t distribution.
Limits and warnings
- A failed test shows that the traces depend on the data at some point. It does not show that a key can be recovered, and a passed test does not show that none can be. It is a screening step before attacks, not a certificate.
- The test is univariate: each sample point on its own. Leakage spread across several points, for example a masked device whose shares sit at different times, needs a multivariate second-order test that this tool does not run.
- The fixed-versus-random test is non-specific. It detects any data dependence, including that of harmless intermediate values. The standard ISO/IEC 17825 sets out the test vectors and the pass rules for a formal evaluation; this tool does not replace that process.
- With thousands of points, even an ideal device will show some |t| above 4.5 now and then. Use two independent sets, or repeat the capture, before calling a point a leak.
- The second-order test here uses sums from one set of traces and the group means estimated from the same traces; for very small groups its p-values are only a guide.
Sources
The methods and default figures rest on these public sources. The wording here is our own; read the source for its full text.
- A testing methodology for side-channel resistance validation, NIST Non-Invasive Attack Testing Workshop (Goodwill, Jun, Jaffe and Rohatgi), 2011. The fixed versus random Welch t-test and the 4.5 threshold.
- Leakage Assessment Methodology: A Clear Roadmap for Side-Channel Evaluations (opens in a new tab), CHES 2015, Springer LNCS 9293 (Schneider and Moradi), 2015. Higher-order tests on centred, squared values and the one-pass computation of the moments.
- ISO/IEC 17825, Testing methods for the mitigation of non-invasive attack classes against cryptographic modules (opens in a new tab), ISO/IEC, 2016 edition; the 2024 edition replaces it. The formal evaluation method that uses these t-tests. Cited by name only; this tool does not reproduce its text.
- Does My Device Leak Information? An a priori Statistical Power Analysis of Leakage Detection Tests (opens in a new tab), ASIACRYPT 2013, Springer (Mather, Oswald, Bandenburg and Wojcik), 2013. Why the number of traces depends on the effect size and the power.
- NIST/SEMATECH e-Handbook of Statistical Methods, section 1.3.5.3, Two-Sample t-Test for Equal Means (opens in a new tab), National Institute of Standards and Technology, Online edition. Welch's statistic and the Welch-Satterthwaite degrees of freedom.
Where it helps
The lifecycle stages this tool serves: 2 stages in 1 lifecycle.
More safety and security tools
- Hardware safety metrics calculator
- Safety integrity level determination
- Fault tree calculator
- Fault injection campaign sizer
- Safety function failure probability calculator
- Threat risk assessment calculator
- Vulnerability severity calculator
- Software update impact assessment
- SBOM licence checker
- Development assurance objectives checklist
- Medical device classification helper
Method notev1Formula, ranges, sources and limits
Runs the fixed versus random Welch t-test for side-channel leakage and sizes the number of traces needed to detect a given effect at a stated power.
Formula
delta = x − mean; mean = mean + delta / n; M2 = M2 + delta x (x − mean new); sample variance = M2 / (n − 1); t = (mean fixed − mean random) / sqrt(var fixed / n fixed + var random / n random); df = (a + b)^2 / (a^2 / (n fixed − 1) + b^2 / (n random − 1)), where a = var fixed / n fixed and b = var random / n random; p = I(x; df / 2, 1/2) with x = df / (df + t^2); y = (x − mean)^2; mean of y = M2 / n; var of y = (M4 − M2^2 / n) / (n − 1); false alarms expected = points x 2 x (1 − Phi(threshold)); two sets: points x (2 x (1 − Phi(threshold)))^2 / 2; traces per group = 2 x (threshold + z power)^2 / d^2; expected t = d / sqrt(1 / n fixed + 1 / n random); smallest effect = (threshold + z power) x sqrt(1 / n fixed + 1 / n random)
| Input | Unit | Range it holds for | Comes from |
|---|---|---|---|
| Data source | choice | example, summary, traces | Your figure |
| Example device | choice | aes, masked | Your figure |
| Sample points in the example | points | 2 to 200 | Your figure |
| Traces per group in the example | traces | 2 to 20000 | Your figure |
| Noise in the example | standard deviation in trace units | from 0 | Your figure |
| Leakage per bit in the example | trace units per bit | from 0 | Your figure |
| Threshold | absolute t | 1 to 10 | Your figure |
| Standardised effect to detect | standard deviations | 0.001 to 5 | Your figure |
| Power | fraction | 0.5 to 0.9999 | Your figure |
Outputs: Traces needed per group (traces); Traces needed in total (traces); Chance per point of exceeding the threshold with no leak (parts per million); Smallest effect found with the traces in each group (standard deviations); Largest absolute t value, first order (t).
Assumptions
- The fixed and random groups are acquired in an interleaved order, with the same set-up, so drift in the environment affects both groups alike.
- At each sample point the readings in a group are independent and roughly normal, or the groups are large enough for the central limit theorem to hold. The second-order test needs far more traces than the first-order test for the same effect, because it works on squares.
- Example traces are seeded and synthetic: a baseline, leakage proportional to Hamming weights in narrow windows, and Gaussian noise. They teach the method; they are not measurements of any device.
- The sizing treats the effect as the standardised difference of means at one point, with equal group sizes, and uses the normal approximation to the t distribution.
Uncertainty: The t statistics are exact for the data. The sizing is a normal approximation that is close for hundreds of traces or more; it assumes equal groups and an effect confined to one sample point.
Known limits
- A failed test shows that the traces depend on the data at some point. It does not show that a key can be recovered, and a passed test does not show that none can be. It is a screening step before attacks, not a certificate.
- The test is univariate: each sample point on its own. Leakage spread across several points, for example a masked device whose shares sit at different times, needs a multivariate second-order test that this tool does not run.
- The fixed-versus-random test is non-specific. It detects any data dependence, including that of harmless intermediate values. The standard ISO/IEC 17825 sets out the test vectors and the pass rules for a formal evaluation; this tool does not replace that process.
- With thousands of points, even an ideal device will show some |t| above 4.5 now and then. Use two independent sets, or repeat the capture, before calling a point a leak.
- The second-order test here uses sums from one set of traces and the group means estimated from the same traces; for very small groups its p-values are only a guide.
Where the method comes from
Follows Leakage Assessment Methodology: A Clear Roadmap for Side-Channel Evaluations (Schneider and Moradi, CHES 2015) (opens in a new tab). The platform's version departs from it as follows.
- The second-order test is univariate and works on the running central sums of a single set; the two-set confirmation rule and the trace-count sizing are added. Why: Masked designs need a higher-order test, and a team needs to know how many traces to capture.
Reference example, checked by test: Sizing worked by hand from the normal approximation: threshold 4.5, effect 0.1, power 0.99 (z = 2.326348): 2 x (4.5 + 2.326348)^2 / 0.1^2 = 9319.7, so 9,320 traces per group. The two-sided tail beyond 4.5 is 6.795 in a million. With 500 traces per group the smallest effect is (4.5 + 2.326348) x sqrt(2 / 500) = 0.4317. Traces needed per group 9,320; Traces needed in total 18,640; Chance per point of exceeding the threshold with no leak 6.795; Smallest effect found with the traces in each group 0.4317.
Versions
- v1 (2026-10-25): First version, built from the tool idea.
Terms on this page
- Test pattern
- A sequence of input values and expected outputs applied to a chip by a tester to detect manufacturing faults or check a function.
- ISO
- The worldwide federation of national standards bodies that publishes consensus standards across industries, including the automotive functional safety and cybersecurity standards that govern semico...
- IEC
- The Geneva-based international standards organisation for electrical, electronic and related technologies, publishing everything from component symbols and safety requirements to functional safety...
Tell us what you're after and leave your email: we'll connect you with the right person on the Hub.