How to Use Coverage Analysis to Increase Testbench Performance

Graser - Rolls

<1>

Agenda
Overview Multi-Language Simulation Debugging Environment Coverage Analysis New Update Information

<2>

Agenda
Overview Multi-Language Simulation Debugging Environment Coverage Analysis New Update Information

<3>

Incisive Platform Relation

Overview

IUS Coverage

<4>

Incisive Enterprise Simulator

Core Capabilities Incisive Unified Simulator

IUS

- Industry leading, multi-language simulator with ABV - Event-driven native compiled code in a single kernel architecture for highest-performance - Code ,functional coverage and static analysis included

+
SimVision
- GUI for unified debug and analysis - Multi-language debug with schematic tracing

<5>

Incisive Simulation Core

Single kernel architecture
- Mixed languages in a single instruction stream: Verilog, VHDL. - One simulator for all levels of abstraction: behavioral, digital, analog Verilog

VHDL VHDL SystemC PSL/ OVL System Verilog Verilog AMS VHDL AMS

Compile

Compile

Compile

Compile

Compile

Compile

Compile

Compile

Key benefits
- Native compile for high performance - Optimized for small memory footprint - Integrated coverage analysis - One debugging environment across all design disciplines: architecture, HW, SW, mixed language/signal - Open standard interfaces assures design portability
Dynamic Engines Static Engines

Verification Results

Unified Debug and Analysis

Debug and Analysis

Assertion Analysis

Comprehensive Coverage

<6>

Agenda
Overview Multi-Language Simulation Debugging Environment Coverage Analysis New Update Information

<7>

Multi-Language Details
Aspect Oriented Prog.

System-level

Scenarios & Sequences Infinity Minus Built-In High Abstraction Modeling

SystemC 2.1 VHDL 200x

Transactions

C Interfaces Object Oriented Extensions Coverage

e
System Verilog

TA

Dynamic Data Types Port Interfaces TA Recording & Analysis

SystemC 2.1

VHDL 2000 & VHDL 200x OVL/PSL

Signal-level RTL

Constraint Randomization Assertions Clocking Domains

OVL/PSL

OVL/PSL, SVA System Verilog

OVL/PSL

RTL

Static Data Types Concise Language Standard RTL

SystemC 1.0 VHDL 2000 Verilog 2001

Gate-level

OSCI SystemC p1666

VHDL IEEE 1076

<8>

Verilog IEEE 1364

IEEE p1647

How Native Compiled Code is Interleaved

Simulator-X Compile Verilog Compile VHDL Compile AS T Single contiguous code stream VS T . Elaborate
simulatorX simulatorX

VHDL

<9>

VHDL

Update Verilog

Verilog

New irun Front End

Single line activation of IES

Single Step Execution % irun [Sources] top [lib.]cell(.view) [Other_Switches]

< 10 >

Agenda
Overview Multi-Language Simulation Debugging Environment Coverage Analysis New Update Information

< 11 >

Running Simulation on NClaunch

Ncvhdl Refresh Editor Ncvlog Ncelab Irun

File Browser

Library Browser

Console

< 12 >

SimVision Advanced Analysis Debug GUI

Integrated Debugging Environment

SimVision Does it All!

Multi-Language Debugging

HDL Waveforms Signal/Transactions Analog/Digital

Coverage Information

Assertion Browser
< 13 >

Schematic Tracing

Agenda
Overview Multi-Language Simulation Debugging Environment Coverage Analysis New Update Information

< 14 >

Introduce the Coverage Basic

The goal of coverage is to measure how well the testbench verifies the design:
To identify design areas in which to focus verification efforts To estimate the remaining verification effort What do I test next?
Aim tests for area with lowest coverage

Coverage data helps answer these kinds of questions:

Did my last change break the design?

Run a quick regression (knowing what tests cover the changed area)

How can I be absolutely sure?

Run a complete regression (knowing they have high coverage)

When will I be done?

Track your progress over time

Am I done now?
Do the functional and code coverage meet your requirements?

< 15 >

Introduce the Coverage Process

1. Instrument the design Automatic: tools add coverage monitors to inferred coverage points Manual: user adds coverage monitors (PSL, SVA, covergroups) 2. Collect the coverage data Coverage point monitors count occurrences of coverage point hits during simulation 3. Reduce the coverage data Tools bin and cross coverage data to make it more manageable Tools record data to coverage databases for later analysis 4. Analyze the coverage data Tools merge coverage from multiple runs Tools report coverage statistics using user-selected formats User diagnoses reports to determine additional test requirements

< 16 >

Coverage types
Comprehensive coverage include: 1. Code Coverage Analyze HDL code structure which blocks of design code execute. Determine how fully code structure is exercised. 2. Functional coverage Analyze design functionality which functions such as Add and Queue executed. Determine how fully design behavior is exercised.

< 17 >

View Coverage Summary

Start the reporting tool in the graphical mode (iccr -gui) and with the File menu, open the coverage test(s).

< 18 >

View Code Coverage Details

Select the Code/Data tab. Select an instance-based display and expand the instance hierarchy. Double-click an instance to view further details...

< 19 >

Briefly Describe Block Coverage

What is a block?
Normally a set of statements (may be only one) that always execute together.

What does block coverage do?

Monitor and count block entry. Calculate and report block entry statistics.

Why bother with block coverage?

To measure how completely your testbench exercises the design code. It is the most effective (and thus usually the first) code coverage type employed.

< 20 >

Define a Code Block

A block is a sequence of statements that always executes together. Either none or all of the statements in a block execute. A block can contain a single statement. Any construct that breaks execution flow creates a new block, for example:

Block coverage = Line coverage

Construct begin (explicit or implicit) branch statements event and timing control loop statements begin if, elsif, else case item wait for/while loop next, exit VHDL Verilog begin, fork if, else case item wait, @, # for, forever, repeat, while disable

< 21 >

Example Code Blocks

VHDL
begin <block> end block ; if (a=b) then <block> else <block> end if ; case (x) is when 0 -> <block> when others -> <block> end case ; while (a=b) loop <block> end loop ; wait for 10 ns ; <block>

Verilog
begin <block> end if (a==b) <block> else <block> case (x) 0 : <block> default : <block> endcase while (a==b) <block> # 10 ; <block>

< 22 >

View Block Coverage Details

Select to view the block coverage details...

< 23 >

Briefly Describe Expression Coverage

What is an expression? An arithmetic, boolean, or relational expression. What does expression coverage do? Monitor and count operand states. Calculate and report term value coverage. To cover insufficiency of Why bother with expression coverage? block coverage Expression coverage helps explain why the execution thread entered a block. It is the most natural next verification step after you attain high block coverage. Expression coverage involves a multitude of dimensions: - Which statements and operators to include? - How to score the expressions?

< 24 >

View Expression Coverage Details

Select to view the expression coverage details...

< 25 >

Briefly Describe Toggle Coverage

What does toggle coverage do? Collect and report design signal toggle activity What is a design signal toggle? Normally a binary transition (and return after a finite delay) of a DUT signal. Signals may transit through (but not terminate at) an unknown state. Why bother with toggle coverage? Its the only code coverage available for a gate-level netlist Verify that design interconnect is connected and wiggles

Netlist code coverage

< 26 >

View Toggle Coverage Details

Select to view the toggle coverage details...

< 27 >

Briefly Describe FSM Coverage

Users generally look at:
- State coverage what states visited - Transition coverage what state transitions occurred - Arc coverage why each state transition occurred

FSM coverage needs to address several issues:

- Recognize FSMs coded using different styles - Recognize FSMs surrounded by other code - Allow users to tag FSMs

< 28 >

View FSM Coverage Details

This FSM coverage details window is displaying uncovered state information.

< 29 >

View FSM Coverage Details

This FSM coverage details window is displaying uncovered transitions with the input conditions that would have caused them.

< 30 >

Code Coverage Collection Process

To collect code coverage data, you compile, elaborate, and simulate.

Insert coverage configuration by giving options Elaborate the Design ncelab top_level_unit [coverage options] [other_options] Coverage Option
-coverage <coverage_types>
ICC does block, expression, FSM, toggle, and functional coverage

Description
Coverage types {b e f t u a}
To select all coverage use b:e:f:t:u or all.

-covfile <coverage_configuration_file> -covdut <DUT_module_or_entity>

Coverage configuration file

Provide this option once for each configuration file

Restrict coverage hierarchies

Provide this option once for each hierarchy.

< 31 >

Code Coverage Collection Process

To collect code coverage data, you compile, elaborate, and simulate.

Simulate the Design

irun source_files(s) ncsim snapshot [coverage_options] [other_options] [coverage_options] [other_options]

Coverage Option
-covworkdir directory -covdesign directory -covtest directory -covoverwrite -covnomodeldump -input tcl_script

Description
Specify coverage work directory Specify coverage design directory Specify coverage test directory Overwite existing test directory Suppress dumping coverage model Input Tcl command script

Default
cov_work/ Design/ Test/

< 32 >

Code Coverage Reporting Process

To report code coverage, you:
Step 1. Launch 2.Load 3.List 4.Operate 5.Select 6.Mark 7.Report
iccr load_test <test_spec(s)> test_spec ::= [[workdir/]design/]test

Analyzer Command

list_coverage_files list_coverage [-instance |-module |-entity] <list> merge [-code] [-toggle] [-fsm] [-functional] {<test_specs> | -testfile <filename>} output <test> [message] (de)select_coverage [-betsa] [-instance |-module |-entity] <list> mark {-ignore | -covered} {-instance |-module |-entity} {-b [-recursive] <name> | <indices> -e <name> <indices> <indices>}
Indices ::= { n[,] | n1-n2[,] | * }

report_summary [-instance |-module |-entity] [-hidezero] [-nocgopt] [-betsafd] <list> report_detail [-instance | -module |-entity ] [-covered | -uncoverd | -both | marked ] [ -nocgopt ] [-nosource] [-nocompact] [-betdafd ] <list> report_html [-instance | -module |entity] [-output <rptname>] [-covered |uncovered | -both | -marked] [-nocgopt ] [-nocgopt ] [-nosource ] [nocompact ] [betsafd] <list>

< 33 >

Compare Functional Coverage to Code Coverage

Code Coverage What -Line , statement , Block , Expression , Path -FSM state, Arcs -Automated -Good first-level metric -No correlation to desired functionality -Necessary but not sufficient Functional Coverage -Data: Value, Range -Control: Conditions, Sequences, Protocols -Closely correlated to desired functionality -Additional effort

Pros

Cons

Conclusion

-Necessary to achieve efficient functional closure

< 34 >

Compare Functional Coverage to Code Coverage

Code Coverage Functional Coverage

Testbench
DUT

Testbench
DUT

Code Coverage Stimulus generation Protocol checking Data checking Protocol coverage Test plan coverage

Code Coverage

SVA,PSL

Stimulus generation Protocol coverage Protocol checking Test plan coverage Data checking

< 35 >

Functional Coverage intro

Functional coverage focuses on the functional aspects of your design:
User-specified scenarios, corner cases, protocols, etc. As apposed to code coverage (which blindly focuses on individual items) User must be familiar with the design!

Functional coverage is in two flavors:

Control-oriented functional coverage Data-oriented functional coverage

< 36 >

Briefly Describe Control-Oriented Functional Coverage

Control-oriented functional coverage focuses on design behavior:
Sequences of actions Design protocols

Control-oriented functional coverage slightly overlaps but greatly extends code coverage capabilities, with:
Arbitrarily complex sequences Temporal logic

< 37 >

Briefly Describe Data-Oriented Functional Coverage

Data-oriented functional coverage focuses on design data:
Variable values and range FSM states and transitions

Data-oriented functional coverage somewhat overlaps (but also extends) code coverage capabilities, with:
Conditional scoring Cross-products

< 38 >

Code Coverage Collection Process

To collect code coverage data, you compile, elaborate, and simulate.

Insert coverage configuration by giving options Elaborate the Design ncelab top_level_unit [coverage options] [other_options] Coverage Option
-coverage <coverage_types>
ICC does block, expression, FSM, toggle, and functional coverage

Description
Coverage types {b e f t u a}
To select all coverage use b:e:f:t:u or all.

-covfile <coverage_configuration_file> -covdut <DUT_module_or_entity>

Coverage configuration file

Provide this option once for each configuration file

Restrict coverage hierarchies

Provide this option once for each hierarchy.

< 39 >

Code Coverage Collection Process

To collect code coverage data, you compile, elaborate, and simulate.

Simulate the Design

irun source_files(s) ncsim snapshot [coverage_options] [other_options] [coverage_options] [other_options]

Coverage Option
-covworkdir directory -covdesign directory -covtest directory -covoverwrite -covnomodeldump -input tcl_script

Description
Specify coverage work directory Specify coverage design directory Specify coverage test directory Overwite existing test directory Suppress dumping coverage model Input Tcl command script

Default
cov_work/ Design/ Test/

< 40 >

View Functional Coverage Summary

Start the reporting tool in the graphical mode (iccr -gui) and with the File menu, open the coverage test(s).

< 41 >

View Data-Oriented Coverage

Select the Functional tab and expand the Data-oriented Coverage...

< 42 >

View Control-Oriented Coverage

Select the Functional tab and expand the Controloriented Coverage...

< 43 >

Agenda
Overview Multi-Language Simulation Debugging Environment Coverage Analysis New Update Information

< 44 >

Deselect Coverage with File

We can effectively exclude our model by typing file name following by the command, which means if you want to exclude the CPU in your design then just type the CPUs file name behind and do not have to alter each single module and merge into it. This utility is not yet supported for the functional coverage. This new utility is released in the newest IUS82 s17 version.

Configuration command : deselect_coverage [-betf] [[-module] list | instance list | -file list]

< 45 >

Deselect Coverage with File

With deselect coverage command it could ignore certain files as the blackbox to increase the total coverage rate
< 46 >

Statement Coverage
In a design, an unverified block with MORE statements is likely to have MORE ERRORS than the one with a single statement. A block coverage report, by default, does not include information on the number of statements within a block
set_statement_score

Command:

With this command, scoring of statements is enabled, and the statement coverage information is also included in the block coverage report.

< 47 >

Statement Coverage
The following report is generated from the code after enabling branch and statement coverage.

In the above report, column #Stmts shows the number of statements in the corresponding block.
< 48 >

Thank You !!!

< 49 >