bench(3tcl) | Benchmarking/Performance tools | bench(3tcl) |
bench - bench - Processing benchmark suites
package require Tcl 8.2
package require bench ?0.4?
::bench::locate pattern paths
::bench::run ?option value...? interp_list file...
::bench::versions interp_list
::bench::del bench_result column
::bench::edit bench_result column newvalue
::bench::merge bench_result...
::bench::norm bench_result column
::bench::out::raw bench_result
This package provides commands for the execution of benchmarks written in the bench language, and for the processing of results generated by such execution.
A reader interested in the bench language itself should start with the bench language introduction and proceed from there to the formal bench language specification.
The command resolves soft links to find the actual executables matching the pattern. Note that only interpreters which are marked as executable and are actually executable on the current platform are put into the result.
It is assumed that the contents of the files are written in the bench language.
The available options are
If no paths were specified the system will behave as before. If one or more paths are specified, say N, each of the specified interpreters will be invoked N times, with one of the specified paths. The chosen path is put into the interpreters' auto_path, thus allowing it to find specific versions of a package.
In this way the use of -pkgdir allows the user to benchmark several different versions of a package, against one or more interpreters.
Note: The empty string is allowed as a path and causes the system to run the specified interpreters with an unmodified auto_path. In case the package in question is available there as well.
The command uses builtin command info patchlevel to determine the version of each interpreter.
The benchmark results are in the format described in section Result format.
The column is identified by an integer number.
The benchmark results are in the format described in section Result format.
The column is identified by an integer number.
All benchmark results are in the format described in section Result format.
The baseline against which the command normalizes are the timing results in the chosen column. This means that after the normalization the values in this column are all 1, as these benchmarks are neither faster nor slower than the baseline.
A factor less than 1 indicates a benchmark which was faster than the baseline, whereas a factor greater than 1 indicates a slower execution.
The benchmark results are in the format described in section Result format.
The column is identified by an integer number.
For other formatting styles see the packages bench::out::text and bench::out::csv which provide commands to format benchmark results for human consumption, or as CSV data importable by spread sheets, respectively.
Complementary, to read benchmark results from files, sockets etc. look for the package bench::in and the commands provided by it.
After the execution of a set of benchmarks the raw result returned by this package is a Tcl dictionary containing all the relevant information. The dictionary is a compact representation, i.e. serialization, of a 2-dimensional table which has Tcl interpreters as columns and benchmarks as rows. The cells of the table contain the timing results. The Tcl interpreters / columns are identified by their paths. The benchmarks / rows are identified by their description.
The possible keys are all valid Tcl lists of two or three elements and have one of the following forms:
The associated value is the version of the Tcl interpreter.
The associated value is irrelevant, and set to the empty string.
The associated value is either one of several special result codes, or the time it took to execute the benchmark, in microseconds. The possible special result codes are
This document, and the package it describes, will undoubtedly contain bugs and other problems. Please report such in the category bench of the Tcllib Trackers [http://core.tcl.tk/tcllib/reportlist]. Please also report any ideas for enhancements you may have for either package and/or documentation.
When proposing code changes, please provide unified diffs, i.e the output of diff -u.
Note further that attachments are strongly preferred over inlined patches. Attachments can be made by going to the Edit form of the ticket immediately after its creation, and then using the left-most button in the secondary navigation bar.
bench_intro, bench_lang_intro, bench_lang_spec, bench_read, bench_wcsv, bench_wtext
benchmark, merging, normalization, performance, testing
Benchmark tools
Copyright (c) 2007-2008 Andreas Kupries <andreas_kupries@users.sourceforge.net>
0.4 | tcllib |