Covered - Verilog Code Coverage Analyzer
covered [global_options] score
[options]
covered [global_options] merge [options]
existing_database database_to_merge+
covered [global_options] report [options]
database_file
covered [global_options] rank [options]
database_to_rank database_to_rank+
covered [global_options] exclude [options]
exclusion_id+ database_file
Covered is a Verilog code coverage analysis tool that can
be useful for determining how well a diagnostic test suite is covering the
design under test. Covered reads in the Verilog design files and a VCD, LXT2
or FST formatted dumpfile from a diagnostic run and generates a database
file called a Coverage Description Database (CDD) file, using the score
command. Covered's score command can alternatively be used to generate a CDD
file and a Verilog module for using Covered as a VPI module in a testbench
which can obtain coverage information in parallel with simulation (see
USING COVERED AS A VPI MODULE). The resulting CDD file can be merged
with other CDD files from the same design to create accummulated coverage,
using the merge command. Once a CDD file is created, the user can use
Covered to generate various human-readable coverage reports in an ASCII
format or use Covered's GUI to interactively look at coverage results, using
the report command. If uncovered coverage points are found that the user
wants to exclude from coverage, this can be handled with either the
command-line exclude command or within the GUI. When multiple CDD files are
created from the same design, the user may obtain a coverage ranking of
those CDD files to determine an ideal order for regression testing as well
as understand which CDD files can be excluded from regressions due to their
inability to hit new coverage points. Additionally, as part of Covered's
score command, race condition possibilities are found in the design files
and can be either ignored, flagged as warnings or flagged as errors. By
specifying race conditions as errors, Covered can also be used as a race
condition checker.
These options are placed immediately after the keyword
covered in the command-line. They can be used for any command (with
the exception of -v and -h) and have the same effect in each
case.
- -B
- Obfuscate. Obfuscates all design-sensitive names before outputting in
user-readable format. This option is useful when sharing output with the
developers of Covered for debugging purposes.
- -D
- Debug. Display information helpful for debugging tool problems. Note: This
option is now only available when covered is built with the --enable-debug
configuration option.
- -h
- Help. Display this usage information.
- -P [filename]
- Profiling mode. Turns on internal source code profiler that will produce a
profiling report of the run command to either the specified
filename or, if no filename is present, to a file called
covered.prof. This option is only available if the
--enable-profiling configuration option was specified when Covered
was built.
- -Q
- Quiet mode. Causes all output to be suppressed.
- -T
- Terse mode. Causes all output to be suppressed with the exception of
warning messages and the Covered header information.
- -v
- Version. Display current Covered version.
- score
- Parses Verilog files and VCD/LXT2/FST dumpfiles to create database file
used for merging and reporting.
- merge
- Merges two or more database files into one.
- report
- Generates human-readable coverage reports from database file or starts the
coverage report GUI.
- rank
- Generates a report that specifies an ideal order to run regressions and
specifies CDD files that do not add new coverage information (and can,
therefore, be excluded from regressions, if desired).
- exclude
- Allows one or more coverage points (identified with exclusion IDs in a
report generated with the -x option) to have their exclusion property
toggled (include to exclude or exclude to include) and, if the exclusion
property is set to exclude, optionally allow a reason for the exclusion to
be associated with it and stored in the specified CDD file.
The following options are valid for the score command:
- -A ovl
- Causes OVL assertions to be used for assertion coverage. This flag must be
given to the score command if assertion coverage metrics are needed in the
report command.
- -cdd
database
- Same as the -o option. Useful when CDD file being scored is an input to
the score command.
- -cli
[filename]
- Causes the command-line debugger to be used during VCD/LXT2/FST dumpfile
scoring. If filename is specified, this file contains information
saved in a previous call to savehist on the CLI and causes the history
contained in this file to be replayed prior to the CLI command prompt. If
filename is not specified, the CLI prompt will be immediately
available at the start of simulation. This option is only available when
Covered is configured with the --enable-debug option.
- -conservative
- If this option is specified, any logic blocks that contain code that could
cause coverage discrepancies leading to potentially inaccurate coverage
results are removed from coverage consideration. See User's Guide for more
information on what type of code can lead to coverage inaccuracies.
- -dumpvars
[filename]
- If this option is specified without the -vcd or -lxt options, the design
is parsed, a CDD file is created and a top-level Verilog module file named
filename (if this value is specified) or "covered_dump.v"
(if filename is not specified) is created. This file is used in the
compilation of the simulator to create a dumpfile that is optimized for
obtaining coverage for the specified CDD file. If either the -vcd or -lxt
options are specified, this option has no effect. See the User's Guide for
more information on how to use this option.
- -D
define_name
- Defines the specified name to 1.
- -D
define_name=value
- Defines the specified name to the specified value.
- -e block_name
- Name of module, task, function or named begin/end block to not score.
Causes all subblocks in the Verilog tree under this block to also not be
scored.
- -ea
- Excludes all always blocks from being considered for coverage.
- -ec
- Excludes all continuous assignments from being considered for
coverage.
- -ei
- Excludes all initial blocks from being considered for coverage.
- -ef
- Excludes all final blocks from being considered for coverage.
- -ep [name]
- Excludes all code found between '// coverage off' and '// coverage on'
pragmas embedded within the design. If name is specified, the
keyword in the pragma will be changed to that name instead of the default
"coverage", allowing the user to change the look of the pragma
if it conflicts with any other tools.
- -F
module_name=[in_expr,]out_expr
- Indicates to the parser where to find the FSM located in module
module_name which has an input state expression called
in_expr and output state expression called out_expr. If
in_expr is not specified, out_expr is used as both the input
and output state expression.
- -f filename
- Name of file containing additional arguments to parse.
- -fst
filename
- Name of FST dumpfile to score design with. If -vcd, -lxt or this option is
not used, Covered will only create an initial CDD file from the design and
will not attempt to score the design.
- -g
[module_name=](1|2|3)
- Allows the user to limit the parser to a specific generation of the
Verilog standard for a specific module or the entire design, where
1=Verilog-1995, 2=Verilog-2001, 3=SystemVerilog. If module_name= is
specified, the parser uses the specified Verilog standard for that module
only, allowing the rest of the design to use the global standard. By
default, the global standard is set to the most recent version (3). This
option can be specified more than once for a given call to the score
command; however, if the -g option specifies more than global value (i.e.,
without the module= prefix), only the last option value will be
used.
- -h
- Displays this help information.
- -I directory
- Directory to find included Verilog files.
- -i
instance_name
- Verilog hierarchical reference to the module that is at the top of the
tree to be scored. This option is necessary if module to verify coverage
is not the top-level module in the design. If not specified, -t
value is used.
- -lxt
filename
- Name of LXT2 dumpfile to score design with. If -vcd, -fst or this option
is not used, Covered will only create an initial CDD file from the design
and will not attempt to score the design.
- -m message
- Allows the user to specify information about this CDD file. This
information can be anything (messages with whitespace should be surrounded
by double-quotation marks), but may include something about the simulation
arguments to more easily link the CDD file to its simulation for purposes
of recreating the CDD file.
- -o database
- Name of database to write coverage information to. If not specified, the
output database filename will be "cov.cdd".
- -p filename
- Overrides default filename used to store intermediate preprocessor
output.
- -P
parameter_scope=value
- Performs a defparam on the specified parameter with value.
- -rS
- When race condition checks are violated, the offending logic blocks are
removed from coverage consideration and all output is suppressed regarding
the race condition violation. See user documentation for more information
about race condition checking usage.
- -rW
- When race condition checks are violated, the offending logic blocks are
removed from coverage consideration and the race condition violation is
output. This is the default behavior for race condition handling. See user
documentation for more information about race condition checking
usage.
- -rE
- When race condition checks are violated, the reason is output and scoring
ends immediately. See user documentation for more information about race
condition checking usage.
- -rI[=module
name]
- If module name is not specified, race condition checking is skipped
altogether for the entire design. If module name is specified, race
condition checking is skipped for the specified module. See user
documentation for more information about race condition checking
usage.
- -rP[=name])
- Uses embedded pragmas for ignoring certain code from race condition
checking consideration (if name is specified it is used as the
pragma keyword). See user documentation for more information about race
condition checking usage.
- -S
- Outputs simulation statistics after simulation has completed. This
information is currently only useful for the developers of Covered.
- -t top-level
module
- Specifies the module name of the top-most module that will be measured.
Note that this module does not need to be the top-most module in the
simulator. This field is required for all calls to the score command.
- -top_ts
timescale
- This option is only valid when the -vpi or -dumpvars options have been
specified. This option allows the user to specify a timescale for the
generated Verilog module created with the -vpi/-dumpvars option. If this
option is not specified, no timescale will be created for the generated
module. The value of timescale is specified as follows:
- (1|10|100)(s|ms|us|ns|ps|fs)/(1|10|100)(s|ms|us|ns|ps|fs)
- If whitespace is needed between the various values, place the entire
contents of timescale in double quotes.
- -ts number
- When scoring occurs, this option allows the user to see how far the
simulator has progressed by outputting the current timestep to standard
output. The value of number specifies how many timesteps are
allowed to be simulated before outputting the current timestep (results in
less calls to output stream).
- -T (min|typ|max)
- Specifies which value to use when encountering a delay expression in the
form: min:typ:max. If this option is not specified, 'typ' select is used
by default.
- -v
filename
- Name of specific Verilog file to score.
- -vcd
filename
- Name of VCD dumpfile to score design with. If -lxt, -fst or this option is
not used, Covered will only create an initial CDD file from the design and
will not attempt to score the design.
- -vpi
[filename]
- If this option is specified without the -vcd, -lxt or -fst options, the
design is parsed, a CDD file is created and a top-level Verilog module
file named filename (if this value is specified) or
"covered_vpi.v" (if filename is not specified) is created
along with a PLI table file called filename.ta b or
"covered_vpi.v.ta b". Both of these files are used in the
compilation of the simulator to use Covered as a VPI module. If either the
-vcd, -lxt or -fst options are specified, this option has no effect.
- -Wignore
- Suppress the output of warnings during code parsing and simulation.
- -y directory
- Directory to find unspecified Verilog files.
- +libext+.extension[+.extension]*+
- Extensions of Verilog files to allow in scoring.
The following options are valid for the merge command:
- -d filename
- Directory to search for CDD files to include. This option is used in
conjunction with the -ext option which specifies the file extension
to use for determining which files in the directory are CDD files.
- -er
(first|last|all|new|old)
- Specifies how to handle exclusion reason resolution. If two or more CDD
files being merged have exclusion reasons specified for the same coverage
point, the exclusion reason needs to be resolved (unless it is the same
string value). If this option is not specified and a conflict is found,
Covered will interactively request input for each exclusion as to how to
handle it. If this option is specified, it tells Covered how to handle all
exclusion reason conflicts. The values are as follows:.br
- first - CDD file that contained the first exclusion reason is
used.
- last - CDD file that contained the last exclusion reason is
used.
- all - All exclusion reasons are used (concatenated).
- new - Use the newest exclusion reason specified.
- old - Use the oldest exclusion reason specified.
- -ext
extension
- Used in conjunction with the -d option. If no -ext options
are specified on the command-line, the default value of '.cdd' is used.
Note that a period (.) should be specified.
- -f
filename
- Name of file containing additional arguments to parse.
- -h
- Displays this help information.
- -m
message
- Allows the user to specify information about this CDD file. This
information can be anything (messages with whitespace should be surrounded
by double-quotation marks).
- -o
filename
- File to output new database to. If this argument is not specified, the
existing_database is used as the output database name.
The following options are valid with the report command:
- -b
- If combinational logic verbose output is reported and the expression is a
vector operation, this option outputs the coverage information on a
bitwise basis.
- -c
- If -v is specified, displays covered metrics only. Default is to
display uncovered information only.
- -d
(s|d|v)
- Level of detail to provide in coverage report information (s = summary, d
= detailed, v = verbose). Default is summary.
- -e
- Outputs all excluded coverage points to the report file along with any
specified exclusion reasons if the -d d or -d v options are
specified.
- -f
filename
- Name of file containing additional arguments to parse.
- -h
- Displays this help information.
- -i
- Provides coverage information for instances instead of module.
- -m
[l][t][c][f][r][a][m]
- Type(s) of metrics to report. l=line, t=toggle, c=combinational logic,
f=FSM state and state transition, r=race conditions, a=assertion,
m=memory. Default is ltcf.
- -o
filename
- File to output report information to. Default is standard output.
- -s
- Suppresses modules/instances that contain no coverage information from
being output to the report. Used to help eliminate potentially meaningless
information from the report.
- -v
- Deprecated. Replaced by '-d d' or '-d v'.
- -view
- Starts the GUI interface for interactive coverage reporting.
- -w
(number)
- Specifies the maximum line width (in characters) that can be used to
output Verilog information. If this option is not specified, all Verilog
code in the report will retain the same formatting as was specified in the
original Verilog code. If this option is specified, Verilog code will be
formatted to use as much of the current line as possible, wrapping text
when the line reaches the maximum line width. The default maximum line
width is 115 characters (this value is used if no number is specified with
the -w option). If a number is specified with the -w option, this value is
used for the maximum line width.
- -x
- Outputs the exclusion IDs of all uncovered and excluded coverage points
within parenthesis before the associated verbose output of the coverage
point. The exclusion IDs can be used to exclude/include coverage points
via the exclude command.
The following options are valid with the rank command:
- -d
filename
- Directory to search for CDD files to include. This option is used in
conjunction with the -ext option which specifies the file extension
to use for determining which files in the directory are CDD files.
- -depth
number
- Specifies the minimum number of needed CDD files to hit each coverage
point. The value of number should be greater than zero. Default is
1.
- -ext
extension
- Used in conjunction with the -d option. If no -ext options
are specified on the command-line, the default value of '.cdd' is used.
Note that a period (.) should be specified.
- -f
filename
- Name of file containing additional arguments to parse.
- -h
- Displays help information for the rank command.
- -names-only
- If specified, outputs only the needed CDD filenames that need to be run in
the order they need to be run. If this option is not set, a report-style
output is provided with additional information. This option is meant to be
useful in scripts that only want CDD filenames to run as output.
- -o
filename
- Name of file to output ranking information to. Default is standard
output.
- -required-cdd
filename
- Name of CDD that should be considered a required CDD to rank (i.e., it
cannot be excluded for any reason).
- -required-list
filename
- Name of a file that contains a list of CDDs that should be considered
required CDDs to rank. The filenames should be separated by whitespace or
newline characters within the file.
- -v
- Causes verbose output to be displayed when the rank command is run. It
outputs diagnostic information about each of the different phases of the
ranking algorithm including run-time, number of CDD files
included/excluded and number of coverage points hit by ranked CDDs during
each phase. This information is meant to be useful for those interested in
the ranking algorithm and its performance.
- -weight-assert
number
- Specifies a relative weighting for assertion coverage used to rank
non-unique coverage points. The value of number is relative to the
values used in the -weight-toggle, -weight-memory,
-weight-comb, -weight-fsm and -weight-line rank
command options.
- -weight-comb
number
- Specifies a relative weighting for combinational logic coverage used to
rank non-unique coverage points. The value of number is relative to
the values used in the -weight-toggle, -weight-memory,
-weight-assert, -weight-fsm and -weight-line rank
command options.
- -weight-fsm
number
- Specifies a relative weighting for FSM state and state transition coverage
used to rank non-unique coverage points. The value of number is
relative to the values used in the -weight-toggle,
-weight-memory, -weight-comb, -weight-assert and
-weight-line rank command options.
- -weight-line
number
- Specifies a relative weighting for line coverage used to rank non-unique
coverage points. The value of number is relative to the values used
in the -weight-toggle, -weight-memory, -weight-comb,
-weight-fsm and -weight-assert rank command options.
- -weight-memory
number
- Specifies a relative weighting for memory coverage used to rank non-unique
coverage points. The value of number is relative to the values used
in the -weight-toggle, -weight-line, -weight-comb,
-weight-fsm and -weight-assert rank command options.
- -weight-toggle
number
- Specifies a relative weighting for toggle coverage used to rank non-unique
coverage points. The value of number is relative to the values used
in the -weight-memory, -weight-line, -weight-comb,
-weight-fsm and -weight-assert rank command options.
The following options are valid with the exclude command:
- -f
filename
- Specifies the name of a file that contains more options to the exclude
command. This option may be specified as many times as necessary for a
single call to the exclude command.
- -h
- Generates usage information for the exclude command.
- -m
- Allows for an exclusion message to be associated with any coverage points
going from the included state to the excluded state. For each coverage
point that meets this requirement, the user will be prompted to input a
reason. The reason may be any length and any number of lines; however, all
formatting characters (i.e., newlines, tabs, extra spaces, etc.) will be
removed and replaced with a single space when it is later displayed. To
end the input of a message, hit a return, enter a single period (.)
character and hit return again. The final period character will not be
part of the exclusion message.
- -p
- Causes all specified coverage points to print their current exclusion
status and exclusion reason (if one exists for the excluded coverage
point) to standard output. If this option is specified, the -m
option will be ignored.
- In addition to using Covered's
score command to parse a VCD, LXT2 or FST file to abstract coverage
information, Covered may also be used as a VPI module within a simulator to
extract this information. The advantages to using Covered as a VPI over a
dumpfile reader include the following. First, VCD files can be extrememly
large, especially for long simulations, using up valuable disk space.
Second, if you are using a simulator that dumps files in a different format
than VCD, LXT2 or FST and you want to convert these dump file types to one
of these versions, the cost of disk space and time can make creating
dumpfiles that Covered requires undesirable. Additionally, though using
Covered as a VPI module will slow down your simulation speed, it is most
likely that the total time spent simulating your design and scoring the
design in one step will be shorter than doing so in two steps. As a result,
Covered's configure utility can generate VPI-ready libraries for the
following free and commercial simulators (Icarus Verilog, CVER and
VCS).
- To automatically build the
VPI-ready library files when generating Covered from source, simply specify
one or more of the following when running the "configure" utility
in the base Covered directory: --with-iv=<Icarus Verilog install
path>, --with-vcs=<VCS include path>, --with-cver=<CVER include
path>. After Covered has been configured, simply type 'make' and 'make
install'. This will install the VPI-ready library files in the installation
libexec directory (by default this path will be
/usr/local/libexec).
- Before you are ready to
compile the design, you must first create a CDD file, a top-level Verilog
file, and a PLI table file (the last file is only needed for the VCS
compiler). This is done by specifying the -vpi (filename)
option to Covered's score command. If no filename is specified after -vpi,
the files covered_vpi.v and covered_vpi.ta b will be created along with the
generated coverage file. Note that this step only needs to be performed once
unless the design files change. You are now ready to compile the
simulator.
- If you are compiling an Icarus
Verilog simulation, simply add '-m /usr/local/libexec/covered.vpi
covered_vpi.v' to the 'iverilog' command-line. Once compilation is complete,
run the generated executable file as you normally would.
- If you are compiling a CVER
simulation, simply add
'+loadvpi=/usr/local/libexec/covered.cver.so:vpi_compat_bootstrap
covered_vpi.v' to the 'cver' command-line.
- If you are compiling a VCS
simulation, simply add '+vpi -load
/usr/local/libexec/covered.vcs.so:covered_register covered_vpi.v' to the
'vcs' command-line. Once compilation is complete, run the generated
executable file as you normally would.
- If you are compiling a
NC-Verilog simulation, switch to NC-Verilog's irun command to load the
covered shared object: '-loadvpi
/usr/local/libexec/covered.ncv.so:covered_register' and enable all access
with '-access +rwc'. You can hardcode the $covered_sim call into your RTL or
you can run it dynamically using the CLI, by adding the -input input.tcl
switch to irun. Where the input.tcl file looks like the following and tb.dut
is the coverage instance:
-
call -systf {$covered_sim} {"scored.cdd"} tb.dut
run
- There are two plusargs that
can be passed to the generated executable when it is run that Covered will
parse. The '+covered_cdd=<filename>' option will cause Covered to
output the scored design contents to the CDD file specified by
<filename>. This allows multiple runs of the simulator to generate
several different CDD files without needed a recompile to occur. The
'+covered_debug' option will cause Covered to dump a lot of excessive output
about its internal run-time state during simulation. This output will only
be generated if Covered was configured with the --enable-debug option. This
plusarg option should not be used by regular users as it is primarily
intended to aid the developers of Covered in debugging.
Trevor Williams <phase1geo@gmail.com>
For more information on how to use the Covered code coverage tool,
please consult the User's Guide included with this release at
/usr/local/share/covered/doc/html/index.html.