Coverage closure tracker
Track code, functional and assertion coverage goals for each block over time, and project when verification will close from the trend.
Digital and physical designFor membersHow it worksSourcesWhere it helps
Register or sign in to use this tool
The working tool, its inputs and its results are for registered members. Registration is free and takes a minute. You come back to this page afterwards.
How it works
Coverage closure is the work of bringing every coverage metric of every block up to its goal before sign-off. This tracker keeps each regression's results, shows where every block stands against its goals, fits the trend of recent regressions and projects the date each metric reaches its goal, so you can see early which blocks will miss the target date.
- Set the goals and weights. Give a goal in percent for each of line, branch, toggle, FSM, functional and assertion coverage. Leave a goal empty to stop tracking that metric. Weights say how much each metric counts in the score; functional and assertion coverage carry more than code coverage because they show that the intended behaviour was exercised, where code coverage only shows that the code was run.
- Log each regression. One row for each block and regression: the date, the block, the percentages and the number of open and waived holes. Leave a metric empty when it does not apply or was not collected (for example FSM coverage in a block with no state machine). The latest row of a block, by date, gives its current figures.
- Score against the goals. Progress on a metric is its latest coverage divided by its goal, never more than one. A block's score is the weighted average of that progress over its measured metrics; the overall score is the average of the block scores.
progress = min(1, coverage / goal); score = sum(weight x progress) / sum(weight) - Fit the trend. The linear model fits a least-squares line through the last N regressions of a metric and projects the day it reaches the goal. The saturating model assumes the shortfall from 100% shrinks by a constant fraction in each interval, the way real coverage slows near the end, so it fits a straight line to ln(100 − coverage) and projects the day the fitted shortfall falls to the goal's shortfall (at least the tolerance, so a goal of 100% is reached at 100 minus the tolerance). Both are shown for each metric; the one you choose decides the status.
linear: days = (goal − fitted now) / slope; saturating: days = ln(shortfall now / goal shortfall) / k, where shortfall = 100 − coverage - Judge against the target date. A block is closed when every measured metric meets its goal. Otherwise it is on track if every unmet metric is projected to close on or before the target date, and at risk if any is projected later, never reaches its goal on the trend (flat or falling), or the target date has already passed. The days late are the latest projected closure minus the target date.
Assumptions
- Goals and weights are editorial defaults. Replace them with the sign-off criteria in your verification plan.
- Coverage figures are percentages from 0 to 100 as your tools report them, after exclusions. Waived holes are counted separately and are not added to the percentages.
- Open and waived hole counts are a snapshot at each regression; the tracker reports the latest one for each block.
- A trend continues at the pace of the recent regressions. Closure work is rarely smooth: the last few percent usually take longer, and a new feature or a design change can lower coverage.
- Dates are calendar days. Today's date is an input so the results do not depend on the clock; leave it empty to use the current date.
Limits and warnings
- A projection from fewer than four or five regressions is a rough guide; the tool warns when a trend explains under half of the variation.
- The fit uses each metric on its own. It does not know about planned test-bench work, tool-flow changes or hole analysis that will move several metrics at once.
- Coverage that is not reproducible between tool versions, or merged across simulators, needs care before regressions are compared. Log results from one consistent flow.
- 100% coverage is not proof of correctness. Functional coverage is only as good as the coverage model, which comes from the verification plan.
- It does not read coverage databases. Enter the figures by hand or import them as CSV or JSON.
Sources
The methods and default figures rest on these public sources. The wording here is our own; read the source for its full text.
- Coverage Cookbook, Verification Methodology Cookbooks (opens in a new tab), Verification Academy, Siemens EDA, page last updated December 2013; announced 20 November 2012. The kinds of coverage (code and functional), how coverage comes from the specification and test plan, and the closure process. Cited as the reference for the metric definitions.
- Unified Coverage Interoperability Standard (UCIS), Version 1.0 (opens in a new tab), Accellera Systems Initiative, 2 June 2012. The standard data model and API for coverage data (scopes, cover items and the kinds of coverage), which is what coverage tools export and merge.
- Coverage Cookbook Debuts (opens in a new tab), Verification Horizons, Siemens EDA (Dennis Brophy), 20 November 2012. An overview of the cookbook's theory and practice sections.
Where it helps
The lifecycle stages this tool serves: 6 stages in 6 lifecycles.
More digital and physical design tools
- Power and thermal estimator
- Power tree and budget calculator
- Register map builder
- FPGA resource and device sizing estimator
- Logic density estimator
- Interconnect RC calculator
- Electromigration wire width calculator
- Layer density and fill calculator
- Model to silicon correlation viewer
- Hard IP port effort estimator
Terms on this page
- Coverage closure
- The iterative process of running regressions, analysing unhit coverage items, and adding constraints, tests or waivers until the coverage goals in the verification plan are met.
- Signoff
- The final set of golden analyses a design must pass before tapeout, run with foundry-qualified tools and models.
- Regression
- The automated, repeated execution of a large suite of tests across many random seeds and configurations, usually nightly on a compute farm, to detect functional regressions and accumulate coverage....
- FSM
- A sequential logic structure with a defined set of states, a next-state function of the current state and inputs, and an output function.
- Code coverage
- Automatically extracted metrics showing which parts of the RTL source were exercised by simulation, including line, branch, condition, toggle and FSM coverage.
- FSM coverage
- A metric that reports which states of an inferred finite state machine were visited and which legal state transitions were taken, sometimes extended to transition sequences.
- Verification plan
- The living document that decomposes the specification into verifiable features, assigns each a method (simulation, formal, emulation) and an owner, and links features to coverage items.
- Functional coverage
- User-defined measurement of which specified behaviours and value combinations a testbench has actually exercised, written as covergroups, coverpoints and crosses.
Need help acting on this?
Tell us what you're after and leave your email: we'll connect you with the right person on the Hub.