critcl(3tcl) | C Runtime In Tcl (CriTcl) | critcl(3tcl) |
critcl - Critcl - Package Reference
package require Tcl 8.4
package require critcl ?3.1.17?
package require platform ?1.0.2?
package require md5 ?2?
::critcl::ccode text
::critcl::ccommand tclname cfunname
::critcl::ccommand tclname arguments body ?option value...?
::critcl::cdata tclname data
::critcl::cconst tclname resulttype value
::critcl::cdefines definitions ?namespace?
::critcl::cproc name arguments resulttype body ?option value...?
::critcl::cproc name arguments resulttype
::critcl::cinit text externals
::critcl::include path
::critcl::api import name version
::critcl::api function resulttype name arguments
::critcl::api header ?pattern...?
::critcl::api extheader ?file...?
::critcl::license author ?text...?
::critcl::summary text
::critcl::description text
::critcl::subject ?key...?
::critcl::meta key ?word...?
::critcl::meta? key
::critcl::buildrequirement script
::critcl::cheaders ?arg...?
::critcl::csources ?pattern...?
::critcl::clibraries ?arg...?
::critcl::source path
::critcl::tsources pattern...
::critcl::owns pattern...
::critcl::cflags ?arg...?
::critcl::ldflags ?arg...?
::critcl::framework ?arg...?
::critcl::tcl version
::critcl::tk
::critcl::preload lib...
::critcl::debug area...
::critcl::check ?label? text
::critcl::checklink ?label? text
::critcl::msg ?-nonewline? msg
::critcl::print ?-nonewline? ?chan? msg
::critcl::compiled
::critcl::compiling
::critcl::done
::critcl::failed
::critcl::load
::critcl::config option ?val?
::critcl::cache ?path?
::critcl::clean_cache ?pattern...?
::critcl::readconfig path
::critcl::showconfig ?chan?
::critcl::showallconfig ?chan?
::critcl::chooseconfig target ?nomatcherr?
::critcl::setconfig target
::critcl::actualtarget
::critcl::buildforpackage ?flag?
::critcl::cnothingtodo file
::critcl::cresults ?file?
::critcl::crosscheck
::critcl::error msg
::critcl::knowntargets
::critcl::sharedlibext
::critcl::targetconfig
::critcl::buildplatform
::critcl::targetplatform
::critcl::cobjects ?arg...?
::critcl::scan path
::critcl::name2c name
::critcl::argnames arguments
::critcl::argcnames arguments
::critcl::argcsignature arguments
::critcl::argvardecls arguments
::critcl::argconversion arguments ?n?
::critcl::argoptional arguments
::critcl::argdefaults arguments
::critcl::argsupport arguments
::critcl::userconfig define name description type ?default?
::critcl::userconfig query name
::critcl::userconfig set name value
::critcl::at::caller
::critcl::at::caller offset
::critcl::at::caller offset level
::critcl::at::here
::critcl::at::get*
::critcl::at::get
::critcl::at::= file line
::critcl::at::incr n...
::critcl::at::incrt str...
::critcl::at::caller!
::critcl::at::caller! offset
::critcl::at::caller! offset level
::critcl::at::here!
::critcl::collect_begin
::critcl::collect_end
::critcl::collect script
::critcl::make path contents
::critcl::has-resulttype name
::critcl::resulttype name body ?ctype?
::critcl::resulttype name = origname
::critcl::has-argtype name
::critcl::argtype name body ?ctype? ?ctypefun?
::critcl::argtype name = origname
::critcl::argtypesupport name code ?guard?
::critcl::argtyperelease name code
::preload library
Welcome to the C Runtime In Tcl, CriTcl for short, a system to build C extension packages for Tcl on the fly, from C code embedded within Tcl scripts, for all who wish to make their code go faster.
This document is the reference manpage for the critcl package. This package is the system's core, i.e. it provides the essential functionality on top of which everything else is built. Its intended audience are mainly developers wishing to write Tcl packages with embedded C code. Some of its sections are however for developers wishing to understand the package internals, and the API it provides to the CriTcl Application. These sections will be marked, allowing package writers to skip them. Users of critcl on the other hand are hereby refered to the applications' manpage, i.e. CriTcl Application. If you are in need of an overview of the whole system instead, please go and read the Introduction To CriTcl.
This package resides in the Core Package Layer of CriTcl.
+----------------+ |Applications | | critcl | | critcl::app | +----------------+ *================* |Core Packages | | critcl | | critcl::util | *================* +----------------+ |Support Packages| | stubs::* | | md5, platform | | ... | +----------------+
A short note ahead of the documentation: Instead of repeatedly talking about "a ".tcl" with embbedded C code", or "a ".tcl" containing critcl commands", etc. we use a shorthand and simply call them ".critcl" files, regardless of their file extension.
The package provides five commands to declare various types of C code fragments. These are:
The result of the command is the empty string.
The result of ::critcl::ccommand itself is the empty string.
The command wraps the body in an invisible C function, compiles it and makes the resulting definition available to all C code fragments declared later on. It itself can assume to have access to all definitions which came before it. See section Runtime Behaviour for more details.
The result of critcl::ccommand itself is the empty string.
The list of arguments contain the names for the four parameters required by a Tcl command function. Superfluous list elements (i.e. beyond the fourth) are ignored. Missing elements (parameters), and empty parameter names are handled by replacing them with standard names. These are, in order of usage
The result of critcl::cdata itself is the empty string.
The result of critcl::cconst itself is the empty string.
The command is similar to critcl::cdata in that it returns a constant value. Contrary to critcl::cdata however it is not restricted to ByteArray results, but accepts all result-types known to critcl::cproc. Its semantics are actually equivalent to
cproc $tclname {} $resulttype "return $value ;"
Contrary to critcl::cproc however it is internally optimized to avoid generating a superfluous C function.
Note that nothing prevents the user from using a C define for the value. Any visible C function is actually also allowed, as long as it does not take arguments.
Please note that this command is only for the lifting of existing C definitions into Tcl. The command does not create the definitions in C. It actually goes so far to check for the presence of the named definitions and not performing the mapping for any which do not exist. Which is sensible, given that non-existing defines have no value which could be used in the mapping.
As these checks are run at the time the embedded C code of a ".critcl" file is actually compiled they have access to and check all C fragments defined with critcl::ccode, plus all the headers it has access to via critcl::cheaders, for that file.
The result of critcl::cproc itself is the empty string.
The only options accepted by this command are:
Note how it is the responsibility of the function to set the interpreter result to an appropriate error message when returning 0.
Attention: The conversion assumes that the value belonged to the function, with an associated reference count, and decrements the reference count to indicate the loss of ownership by the function. This means that it is an error to return a value whose reference count is zero. The system will crash at some point after the return due to reuse of the freed memory.
Note how it is the responsibility of the function to set the interpreter result to an appropriate error message when returning 0.
Attention: Contrary to Tcl_Obj* above this conversion assumes that the returned value has a reference count of 0 and performs no decrement. Returning a value whose reference count is greater than zero will likely cause a memory leak.
The arguments parameter has the overall syntax of a Tcl dictionary value, except that keys (argument names) and values (argument types) are specified in reverse order. Consider the example
int x int y
The argument names must be valid C identifiers.
A limited form of variadic arguments is possible, through optional arguments with default values. For these the argument name is a 2-element list containing the actual name, and the default value. For example, in the declaration
int {x 1}
A caveat! The default value is assigned unconditionally. If a custom argument type uses more complex validation, and the default may be invalid according to it, then the relevant checks have to be done in the procedure body. The argument conversion cannot do it as it is completely bypassed when the argument is not present. Overcoming this requires the separation of argument conversion and validation code.
Below is the list of predefined types legal for arguments, plus the details of their semantics. Note that it is possible to extend this list with custom types if the standard does not support what is needed. See section Advanced: Extending cproc for details.
Further note that the type of the first argument is allowed to be Tcl_Interp*. In that case the argument in question is not counted as an argument of the new Tcl command.
typedef struct critcl_pstring {
Tcl_Obj* o;
char* s;
int len; } critcl_pstring;
typedef struct critcl_list {
Tcl_Obj* o;
Tcl_Obj** v;
int c; } critcl_list;
The function takes an argument of type critcl_bytes containing the original Tcl_Obj* reference of the Tcl argument, plus the length of the byte array and a pointer to the byte data.
typedef struct critcl_bytes {
Tcl_Obj* o;
char* s;
int len; } critcl_list;
Attention: These types are considered DEPRECATED. It is planned to remove their documentation in release 3.2, and their implementation in release 3.3. Their deprecation can be undone if good use cases are shown.
It creates only the shim performing the conversions required by arguments and result.
Both arguments have access to all definitions created by the previously listed commands, regardless of their placement in the ".critcl" file relative to this command. See section Runtime Behaviour for more details.
The C code in text is put into the body of the initialization function of the shared library backing the ".critcl" file, and is executed when this library is loaded into the interpreter.
This code has access to a variable interp of type Tcl_Interp* referencing the Tcl interpreter currently being initialized.
The code in externals on the other hand is placed outside and just before the initialization function, making this is a good place for any external symbols required by initialization function which should not be accessible by any other parts of the C code.
Multiple invokations of this command are allowed, and a later call has access to the information of all preceding ones.
The result of the command is the empty string.
critcl::code {
#include <${path}> }
Newly introduced with critcl version 3 is the support for stubs tables, Tcl's dynamic linking mechanism handling the resolution of symbols between C extensions. We won't go into its details here. See http://wiki.tcl.tk/285 for an introduction in general, and section Stubs Tables for the details of critcl's particular variant.
Critcl supports this via a single command, critcl::api, and its methods.
First, importing stubs tables, i.e. APIs, from another extension:
Important: If foo is a namespaced package name, like, for example "c::stack", then the namespace separators "::" are converted into underscores ("_") in path names, C code, etc.
The first header is expected to contain contains all the necessary stubs table type declarations, mapping macros, etc., and may include package specific headers (See critcl::api header below). This header is included at the beginning of the C code backing the ".critcl" file, and at the beginning of all companion ".c" files. This means that the writer of these files doesn't have to write the necessary #include directory, critcl does it for them.
The second header is expected to contain the stubs table variable definition, and the C code, i.e. definition, of the function to initialize it. This, and a call to this initializer function are added to the ".critcl" file's initialization code.
If the directory containing the aforementioned headers also contains the file "name/name.decls" then it is assumed that this file contains the external representation of the stubs table used to generate the headers. The file is read and the internal representation of the stubs table returned as result of the command, for the importing package to use as it sees fit. If no such file is present the command returns the empty string as its result.
One possible use would be the automatic generation of C code calling on the functions listed in the imported API.
When generating a TEA wrapper the names of the imported APIs are used to declare configure options with which the user can declare a non-standard location for the headers of the API. Any API FOO is translated a single configure option --with-FOO-include.
Second, declaration and export of a stubs table, i.e. API, for the current package, foo:
arguments is a list of C types and associated argument names. Like a dictionary, except that keys (argument names) and values (argument types) are swapped. The resulttype is a C type as well.
All arguments are interpreted as glob pattern and the matching files are copied into the directory containing the generated headers well. As an importing package uses only "fooDecls.h" to access the API this generated header will contain the necessary #include directives to make these companion header files and their declarations available too. Patterns matching no file or non-existing files cause the command to throw an error.
Note that patterns which are not beginning with an absolute path are interpreted relative to the directory containing the current ".critcl" file.
The difference is that these headers will be expected to exist in the external development environment. As such they will be #included in the generated header for the package, but not copied to the package header directory. Nor are they allowed to be glob patterns, as critcl has no context, i.e directory, in which to expand such patterns.
Note that the generated headers for an exported API are included in the package like it is done when importing it somewhere else. To repeat:
The "fooDecls.h" header is included at the beginning of the C code backing the ".critcl" file, and at the beginning of all companion ".c" files. This means that the writer of these files doesn't have to write the necessary #include directory, critcl does it for them.
In mode "compile & run" the generated header files, and their companion headers, if any, are placed in the subdirectory "foo" of the Result Cache. As this location is implicitly added to the include search path any other package importing this API and and build in mode "compile & run" as well will find the these headers.
For mode "generate package" the application was extended with a new option -includedir which specifies the location to place the generated headers in (again in subdirectory "foo" of that path). This path is also be added to the include search paths, ensuring that a package importing an API will find it if the package exporting that API used the same setting for -includedir.
For mode "generate TEA" the static scanner was extended to recognize critcl::api header as a source of companion files. It further uses data about critcl::api import commands to put proper support for --with-foo-include options into the generate "configure(.in)" so that a user may specify custom locations for the headers of any imported API.
Newly introduced with critcl version 3 is support for TEApot meta-data.
While, from the package developer's perspective, some meta data support was already present in critcl v2, through the command ::critcl::license, this was only used to generate and place a file "license.txt" into the built package.
Now critcl supports the declaration of arbitrary meta data, which will be placed into a file "teapot.txt" in a format suitable for use by the TEApot tools [http://docs.activestate.com/activetcl/8.5/tpm/toc.html].
If no text is present the command expects to find a file "license.terms" in the same directory as the ".critcl" file and reads the license from that. Otherwise the license is the joined texts.
This information, the license, is ignored in mode "compile & run", only mode "generate package" uses it. In that case the information is written to a file "license.terms", a sibling to the "pkgIndex.tcl" file in the directory hierarchy of the generated package.
This information is additionally placed into the meta data file "teapot.txt", under the keys as::author and license.
The data specified by this command has priority over any information specified through the generic API ::critcl::meta.
This information is ignored in mode "compile & run", only mode "generate package" uses it. In that case the information is placed into the meta data file "teapot.txt", under the key summary.
The data specified by this command has priority over any information specified through the generic API ::critcl::meta.
This information is ignored in mode "compile & run", only mode "generate package" uses it. In that case the information is placed into the meta data file "teapot.txt", under the key description.
The data specified by this command has priority over any information specified through the generic API ::critcl::meta.
Multiple calls of this command accumulate keywords and phrases.
This information is ignored in mode "compile & run", only mode "generate package" uses it. In that case the information is placed into the meta data file "teapot.txt", under the key subject.
The data specified by this command has priority over any information specified through the generic API ::critcl::meta.
While it is possible to declare information for one of the reserved keys with this command such data is ignored when the final meta data is assembled and written.
Use the commands ::critcl::license, ::critcl::summary, ::critcl::description ::critcl::subject, package require, and package provide to declare data for the reserved keys.
The information for the reserved keys as::build::date and platform is automatically generated by critcl itself.
The envisioned main use is the retrieval of the package's name from within utility packages having to adapt C code templates to their environment. An example of a package using this command for exactly this purpose is critcl::class.
The package provides thirteen commands to control the details of compilation and linking, enabling ".critcl" files to provide custom information about their environment and dependencies.
In important thing to note about all these commands is that the package manages their information on a per-file basis. I.e. information provided by and in a file "FOO.tcl" is kept separate from the information provided by and in a file "BAR.tcl", preventing them from interfering with each other.
The commands are:
All arguments matching the glob pattern -* are forwarded to the compiler's command line when it is invoked for the current ".critcl" file.
All other arguments are interpreted as glob pattern and the matching files are made available to the compiler when it is invoked for the current ".critcl" file. Patterns matching no file or non-existing files cause the command to throw an error.
Note that patterns which are not beginning with an absolute path are interpreted relative to the directory containing the current ".critcl" file.
Note further that this declaration does not cause the specified header files to be automatically #include'd. Inclusion still has to be done via either critcl::include or critcl::ccode, where necessary. It does simply ensure that the compiler will be able to find these files when invoked, by providing the necessary command line flags to extend the compiler's search paths.
Multiple invocations of this command accumulate their information.
All arguments are interpreted as glob patterns. Patterns matching no file or non-existing files cause the command to throw an error. The files matching the patterns are made available to the compiler when it is invoked for the current ".critcl" file. This means that the files in question are compiled together with the ".c" file backing the ".critcl" file into a single object.
Note that patterns which are not beginning with an absolute path are interpreted relative to the directory containing the current ".critcl" file.
Multiple invocations of this command accumulate their information.
All arguments matching the glob pattern -* are forwarded to the linkers's command line when it is invoked for the current ".critcl" file.
All other arguments are interpreted glob patterns. Patterns matching no file or non-existing files cause the command to throw an error. The files matching the patterns are made available to the linker when it is invoked for the current ".critcl" file. This means that the files in question are linked together with the object file backing the ".critcl" file into a single shared library.
Note that patterns which are not beginning with an absolute path are interpreted relative to the directory containing the current ".critcl" file.
Multiple invocations of this command accumulate their information.
The argument is actually considered as glob pattern and all matching files are evaluated. A pattern matching no file or non-existing files cause the command to throw an error.
Note that a pattern not beginning with an absolute path is interpreted relative to the directory containing the current ".critcl" file.
All arguments are considered as glob patterns and the matching files are made available to generated shared library when it is loaded for the current ".critcl" file. Patterns matching no file or non-existing files cause the command to throw an error.
Note that patterns which are not beginning with an absolute path are interpreted relative to the directory containing the current ".critcl" file.
Multiple invocations of this command accumulate their information.
The declared files are sourced after the shared library has been loaded, in the same order they were provided to critcl::tsources.
In that situation it provides the critcl package with information about any files which have to be wrapped and could not be figured out from the previous commands (i.e. critcl::csources, critcl::tsources) because of getting specified dynamically, or getting directly sourced and this not visible to critcl in any other way.
All arguments are forwarded to the compiler's command line when it is invoked for the current ".critcl" file.
Multiple invocations of this command accumulate their information.
All arguments are forwarded to the linkers's command line when it is invoked for the current ".critcl" file.
Multiple invocations of this command accumulate their information.
All arguments are forwarded to the linkers's command line when it is invoked for the current ".critcl" file.
Multiple invocations of this command accumulate their information.
Multiple invocations of this command accumulate their information.
Each library FOO named for preload will be searched at the locations listed below, in the order listed, and the search will stop on the first existing path. Additional notes:
The package provides six commands to control compilation and linking. These are:
Tools like the CriTcl Application are allowed to redefine this procedure to perform their own way of message reporting. The package critcl::app and the application on top print such messages to stdout, for example.
Tools directly using either the critcl package, or the critcl application package are allowed to redefine this procedure to perform their own way of printing.
An example of this is Kettle [https://chiselapp.com/user/andreas_kupries/repository/Kettle/index] where the newest revisions use this to highlight build warnings.
This predicate effectively enables a ".critcl" file used as its own Tcl companion file (see critcl::tsources) to distinguish between sourced by mode "compile & run" for compilation and sourced from either the result of mode "generate package" or during the load phase of "compile & run". In case of the two latter possibilities the result is true, and false for the first.
This enables the Tcl code of a critcl-based package to distinguish between it getting used as a prebuilt package, versus dynamic compile & run, and take action based on that.
Note that this command is only useful from within a ".critcl" file. The result is managed on a per-file basis, like is done for the commands embedding C code and controlling the behaviour of compiler and linker.
See also section Modes Of Operation/Use.
This enables a critcl-based package to check itself for availability and throw an error if it could not be built. Note that the command does not throw such an error itself.
Note further that this command is only useful from within in a ".critcl" file. The result is managed on a per-file basis, like is done for the commands embedding C code and controlling the behaviour of compiler and linker.
It returns true if critcl succeeded in building and loading the package, and false otherwise, i.e. build- or load-failure.
This enables a critcl-based package to to not only check itself for availability and throw an error if it could not be built, but also force an immediate load, circumventing the default behaviour, which is lazy. See also section Runtime Behaviour. Note that the command does not throw any error itself.
Note further that this command is only useful from within in a ".critcl" file. The result is managed on a per-file basis, like is done for the commands embedding C code and controlling the behaviour of compiler and linker.
The package provides a single command for the management of global settings, i.e. configuration options which are independent of any ".critcl" file.
It is expected that this command is irrelevant to anybody just wishing to write a ".critcl" file. It is a management command which is only useful to the CriTcl Application or similar tools.
Side note: This facility requires the use of a tclsh supporting the builtin info frame command. If critcl is run by a tclsh not supporting this no #line directives will be emitted. The command is supported by Tcl 8.5 and higher. It is also supported by Tcl 8.4 provided that it was compiled with the define -DTCL_TIP280. An example of such is ActiveState's ActiveTcl.
Developers of higher-level packages generating their own C code, either directly, or indirectly, by using critcl commands, should also read section Advanced: Location management to see how critcl helps them in generating their directives. Examples of such packages come with critcl itself, see the packages critcl::iassoc and critcl::class.
This package provides two commands for the management of the Result Cache. See that section for background information.
NOTE that these commands are irrelevant to anybody just wishing to write a package using critcl for the C parts. They are management commands which are only useful to the CriTcl Application or similar tools.
The default location is "~/.critcl/[platform::generic]" and usually does not require any changes.
This package provides four commands for the management of the build configuration, i.e. the per-platform information about compilers, linkers, and their commandline options.
NOTE that these commands are irrelevant to anybody just wishing to write a package using critcl for the C parts. They are management commands which are only useful to the CriTcl Application or similar tools.
The twelve commands in this section provide tools like CriTcl Application or similar with deeper access to the package's internals. These commands are irrelevant to anybody just wishing to write a ".critcl" file.
All arguments are interpreted glob patterns. Patterns matching no file or non-existing files cause the command to throw an error. The files matching the patterns are made available to the linker when it is invoked for the current ".critcl" file. This means that the files in question are linked together with the object file backing the ".critcl" file into a single shared library.
Note that patterns which are not beginning with an absolute path are interpreted relative to the directory containing the current ".critcl" file.
Multiple invocations of this command accumulate their information.
The result of the command is a list of 4 elements providing the above mentioned information, in the named order.
The envisioned main use is from within utility packages providing Tcl commands without going through the standard commands, i.e. critcl::ccommand, or critcl::cproc. An example of a package using this command for exactly this purpose is critcl::class.
For the advanced user five commands used inside of critcl::cproc are exposed. These are:
The generated code assumes that the procedure arguments start at index n of the objv array. If this argument is not specified 1 will be assumed.
This package provides one command for the management of package-specific, i.e. developer-specified custom build configuration options.
The type can be either bool, or a list of values.
The description serves as in-code documentation of the meaning of the option and is otherwise ignored. When generating a TEA wrapper the description is used for the configure option derived from the option declared by the command.
A boolean option FOO are translated into a pair of configure options, --enable-FOO and --disable-FOO, whereas an option whose type is a list of values is translated into a single configure option --with-FOO.
It is at this point that definitions and set values are brought together, with the latter validated against the definition.
At the time this command is used only the association between option name and value is recorded, and nothing else is done. This behaviour is necessary as the system may not know if an option of the specified name exists when the command is invoked, nor its type.
Any and all validation is defered to when the value of an option is asked for via ::critcl::userconfig query.
This means that it is possible to set values for any option we like, and the value will take effect only if such an option is both defined and used later on.
First a small introduction for whose asking themselves ´what is location management' ?
By default critcl embeds #line directives into the generated C code so that any errors, warnings and notes found by the C compiler during compilation will refer to the ".critcl" file the faulty code comes from, instead of the generated ".c" file.
Side note: This facility requires the use of a tclsh supporting the builtin info frame command. If critcl is run by a tclsh not supporting this no #line directives will be emitted. The command is supported by Tcl 8.5 and higher. It is also supported by Tcl 8.4 provided that it was compiled with the define -DTCL_TIP280. An example of such is ActiveState's ActiveTcl.
Most users will not care about this feature beyond simply wanting it to work and getting proper code references when reading compiler output.
Developers of higher-level packages generating their own C code however should care about this, to ensure that their generated code contains proper references as well. Especially as this is key to separating bugs concerning code generated by the package itself and bug in the user's code going into the package, if any.
Examples of such packages come with critcl itself, see the implementation of packages critcl::iassoc and critcl::class.
To help such developers eight commands are provided to manage such location information. These are listed below.
A main concept is that they all operate on a single stored location, setting, returning and clearing it. Note that this location information is completely independent of the generation of #line directives within critcl itself.
In terms of ::critcl::at::caller this is equivalent to
critcl::at::caller 0 1
For proper nesting and use it is recommended that such directives are always added to the beginning of a code fragment. This way, should deeper layers add their own directives these will come before ours and thus be inactive. End result is that the outermost layer generating a directive will 'win', i.e. have its directive used. As it should be.
Diversions are for higher-level packages generating their own C code, to make their use of critcl's commands generating Embedded C Code easier.
These commands normally generate all of their C code for the current ".critcl" file, which may not be what is wanted by a higher-level package.
With a diversion the generator output can be redirected into memory and from there on then handled and processed as the caller desires before it is committed to an actual ".c" file.
An example of such a package comes with critcl itself, see the implementation of package critcl::class.
To help such developers three commands are provided to manage diversions and the collection of C code in memory. These are:
The result of the command is the empty string.
Multiple calls are allowed, with each call opening a new nesting level of diversion.
If multiple levels of diversion are open the call only closes and returns the data from the last level.
The command will throw an error if no diversion is active, indicating a mismatch in the pairing of collect_begin and collect_end.
While file generation is related to the diversions explained in the previous section they are not the same. Even so, like diversions this feature is for higher-level packages generating their own C code.
Three examples of utility packages using this facility comes with critcl itself. See the implementations of packages critcl::literals, critcl::bitmap, and critcl::enum.
When splitting a package implementation into pieces it is often sensible to have a number of pure C companion files containing low-level code, yet these files may require information about the code in the main ".critcl" file. Such declarations are normally not exportable and using the stub table support does not make sense, as this is completely internal to the package.
With the file generation command below the main ".critcl" file can generate any number of header files for the C companions to pick up.
The generated file will contain the specified contents.
While the critcl::cproc command understands the most common C types (see section Embedded C Code), sometimes this is not enough.
To get around this limitation the commands in this section enable users of critcl to extend the set of argument and result types understood by critcl::cproc. In other words, they allow them to define their own, custom, types.
If name is declared already an error will be thrown. Attention! The standard result type void is special as it has no accompanying result variable. This cannot be expressed by this extension command.
The body's responsibility is the conversion of the functions result into a Tcl result and a Tcl status. The first has to be set into the interpreter we are in, and the second has to be returned.
The C code of body is guaranteed to be called last in the wrapper around the actual implementation of the cproc in question and has access to the following environment:
resulttype int { Tcl_SetObjResult(interp, Tcl_NewIntObj(rv)); return TCL_OK;
}
resulttype ok { /* interp result must be set by cproc body */ return rv;
} int
If name is declared already an error will be thrown.
The body's responsibility is the conversion of a command's Tcl_Obj* argument into a C value for the underlying function and its storage in a helper variable.
The C code of body is guaranteed to be called inside of a separate C code block (thus allowing the use of local variables) which has access to the following environment:
argtype int { if (Tcl_GetIntFromObj(interp, @@, &@A) != TCL_OK) return TCL_ERROR;
}
argtype float { double t; if (Tcl_GetDoubleFromObj(interp, @@, &t) != TCL_OK) return TCL_ERROR; @A = (float) t;
}
CriTcl can be used in three different modes of operation, called
Of these three Compile & Run came first and is the default when using the package directly. In that case the package collects the C fragments, builds them as needed, and caches the results for quick reuse when the same code is used in the future again.
The second mode, Generate Package, was introduced to enable the creation of (prebuilt) deliverable packages which do not depend on the existence of a build system, i.e. C compiler, on the target machine. This was originally done through the experimental Critbind tool, and is now handled by the CriTcl Application, also named critcl.
Newly introduced with Critcl version 3 is Generate TEA Package. This mode constructs a directory hierarchy from the package which can later be built like a regular TEA package, i.e. using
.../configure --prefix ... make all isntall
Regarding the caching of results please read the section about the Result Cache fore more details.
The default behaviour of critcl, the package is to defer the compilation, linking, and loading of any C code as much as possible, given that this is an expensive operation, mainly in the time required. In other words, the C code embedded into a ".critcl" file is built only when the first C command or procedure it provides is invoked. This part of the system uses standard functionality built into the Tcl core, i.e. the auto_index variable to map from commands to scripts providing them and the unknown command using this information when the command is needed.
A limitation of this behaviour is that it is not possible to just use info commands check for the existence of a critcl defined command. It is also necessary to search in the auto_index array, in case it has not been build yet.
This behaviour can be changed by using the control command critcl::load. When invoked, the building, including loading of the result, is forced. After this command has been invoked for a ".critcl" file further definition of C code in this file is not allowed any longer.
Each ".critcl" file is backed by a single private ".c" file containing that code, plus the boilerplate necessary for its compilation and linking as a single shared library.
The Embedded C Code fragments appear in that file in the exact same order they were defined in the ".critcl" file, with one exception. The C code provided via critcl::cinit is put after all other fragments. In other words all fragments have access to the symbols defined by earlier fragments, and the critcl::cinit fragment has access to all, regardless of its placement in the ".critcl" file.
Note: A limitation of the current system is the near impossibility of C level access between different critcl-based packages. The issue is not the necessity of writing and sharing the proper extern statements, but that the management (export and import) of package-specific stubs-tables is not supported. This means that dependent parts have to be forcibly loaded before their user, with all that entails. See section Runtime Behaviour for the relevant critcl limitation, and remember that many older platforms do not support the necessary resolution of symbols, the reason why stubs were invented for Tcl in the first place.
The compilation of C code is time-consuming critcl not only defers it as much as possible, as described in section Runtime Behaviour, but also caches the results.
This means that on the first use of a ".critcl" file "FOO.tcl" the resulting object file and shared library are saved into the cache, and on future uses of the same file reused, i.e. loaded directly without requiring compilation, provided that the contents of "FOO.tcl" did not change.
The change detection is based MD5 hashes. A single hash is computed for each ".critcl" file, based on hashes for all C code fragments and configuration options, i.e. everything which affects the resulting binary.
As long as the input file doesn't change as per the hash a previously built shared library found in the cache is reused, bypassing the compilation and link stages.
The command to manage the cache are found in section Result Cache Management. Note however that they are useful only to tools based on the package, like the CriTcl Application. Package writers have no need of them.
As a last note, the default directory for the cache is chosen based on the chosen build target. This means that the cache can be put on a shared (network) filesystem without having to fear interference between machines of different architectures.
The audience of this section are developers wishing to understand and possibly modify the internals of critcl package and application. Package writers can skip this section.
It explains how the preloading of external libraries is realized.
Whenever a package declares libraries for preloading critcl will build a supporting shared library providing a Tcl package named "preload". This package is not distributed separately, but as part of the package requiring the preload functionality. This support package exports a single Tcl command
On windows the command will further use the Tcl command ::critcl::runtime::precopy to copy the library to the disk, should its path be in a virtual filesystem which doesn't directly support the loading of a shared library from it.
The command ::critcl::runtime::precopy is provided by the file "critcl-rt.tcl" in the generated package, as is the command ::critcl::runtime::loadlib which generates the ifneeded script expected by Tcl's package management. This generated ifneeded script contains the invocations of ::preload.
The C code for the supporting library is found in the file "critcl_c/preload.c", which is part of the critcl package.
The Tcl code for the supporting runtime "critcl-rt.tcl" is found in the file "runtime.tcl", which is part of the critcl::app package.
The audience of this section are developers wishing to understand and possibly modify the internals of critcl package and application. Package writers can skip this section.
It explains the syntax of configuration files and the configuration keys used by critcl to configure its build backend, i.e. how this part of the system accesses compiler, linker, etc.
It is recommended to open the file containing the standard configurations ("path/to/critcl/Config") in the editor of your choice when reading this section of the documentation, using it as an extended set of examples going beyond the simple defaults shown here.
First, the keys and the meaning of their values, plus examples drawn from the standard configurations distributed with the package. Note that when writing a custom configuration it is not necessary to specify all the keys listed below, but only those whose default values are wrong or insufficient for the platform in question.
gcc -v
gcc -c -fPIC
-DTCL_MEM_DEBUG
-g
-I
-DUSE_TCL_STUBS
-DUSE_TK_STUBS
-DUSE_THREAD_ALLOC=1 -D_REENTRANT=1 -D_THREAD_SAFE=1
-DHAVE_PTHREAD_ATTR_SETSTACKSIZE=1 -DHAVE_READDIR_R=1
-DTCL_THREADS=1
-DNDEBUG
-O2
-o [list $outfile]
NOTE the use of Tcl commands and variables here. At the time critcl uses the value of this key the value of the referenced variable is substituted into it. The named variable is the only variable whose value is defined for this substitution.
.o
gcc -E -dM
gcc -E
gcc -shared
--unresolved-symbols=ignore-in-shared-libs
-Wl,-s
-debug:full -debugtype:cv
-release -opt:ref -opt:icf,3 -ws:aggressive
[info sharedlibextension]
[platform::generic]
The syntax expected from configuration files is governed by the rules below. Again, it is recommended to open the file containing the standard configurations ("path/to/critcl/Config") in the editor of your choice when reading this section of the documentation, using it as an extended set of examples for the syntax>
If the build platform is not a prefix of platform, nor vice versa the whole block is ignored. Otherwise the expression is evaluated via expr, in the same safe interpreter used to run any set and if commands found in the configuration file (see above).
If the expression evaluates to true this configuration block is considered to be the build platform fo the host and chosen as the default configuration. An large example of of this feature is the handling of OS X found in the standard configuration file, where it selects the architectures to build based on the version of the operating system, the available SDK, etc. I.e. it chooses whether the output is universal or not, and whether it is old-style (ix86 + ppc) versus new-style (ix86 32+64) of universality.
The copied values can be overridden later in the configuration file. Multiple copy lines may exist for a platform and be intermixed with normal configuration definitions. If a variable is defined multiple times, the last definition will be used.
also wishing to interface with stubs as they are understood and used by critcl, either by exporting their own stubs table to a critcl-based extension, or importing a stubs table of a critcl-based extension into their own.
To this end we describe the stubs table information of a package foo.
critcl is able to use such a file to give the importing package programmatic access to the imported API, for automatic code generation and the like.
These macros must be defined if, and only if, the C preprocessor macro USE_FOO_STUBS is defined. Package foo does not define this macro, as it is allowed to use the exported functions directly. All importing packages however must define this macro, to ensure that they do not use any of the exported functions directly, but only through the stubs table.
A very reduced, yet also complete example, from a package for low-level random number generator functions can be found at the end of this section.
const FooStubs* fooStubsPtr;
char * Foo_InitStubs(Tcl_Interp *interp, CONST char *version, int exact) {
/*
* Boiler plate C code initalizing the stubs table variable,
* i.e. "fooStubsPtr".
*/
CONST char *actualVersion;
actualVersion = Tcl_PkgRequireEx(interp, "foo", version, exact, (ClientData *) &fooStubsPtr);
if (!actualVersion) { return NULL;
}
if (!fooStubsPtr) { Tcl_SetResult(interp, "This implementation of Foo does not support stubs", TCL_STATIC); return NULL;
}
return (char*) actualVersion; }
The importing package's initialization function must further contain a statement like
if (!Foo_InitStubs (ip, "1", 0)) {
return TCL_ERROR; }
For a complete example of such a header file see below, at the end of this section.
IMVHO critcl's approach is simpler, using only header files found in a single location, vs. header files and static library found in multiple, different locations.
A second simplification is that we avoid having to extend critcl's compiler backend with settings for the creation of static libraries.
Below is a complete set of example header files, reduced, yet still complete, from a package for low-level random number generator functions:
#ifndef rng_DECLS_H #define rng_DECLS_H #include <tcl.h> /*
* Exported function declarations:
*/ /* 0 */ EXTERN void rng_bernoulli(double p, int*v); typedef struct RngStubs {
int magic;
const struct RngStubHooks *hooks;
void (*rng_bernoulli) (double p, int*v); /* 0 */ } RngStubs; #ifdef __cplusplus extern "C" { #endif extern const RngStubs *rngStubsPtr; #ifdef __cplusplus } #endif #if defined(USE_RNG_STUBS) /*
* Inline function declarations:
*/ #define rng_bernoulli (rngStubsPtr->rng_bernoulli) /* 0 */ #endif /* defined(USE_RNG_STUBS) */ #endif /* rng_DECLS_H */
/*
* rngStubLib.c --
*
* Stub object that will be statically linked into extensions that wish
* to access rng.
*/ #ifndef USE_TCL_STUBS #define USE_TCL_STUBS #endif #undef USE_TCL_STUB_PROCS #include <tcl.h> #ifndef USE_RNG_STUBS #define USE_RNG_STUBS #endif #undef USE_RNG_STUB_PROCS #include "rngDecls.h" /*
* Ensure that Rng_InitStubs is built as an exported symbol. The other stub
* functions should be built as non-exported symbols.
*/ #undef TCL_STORAGE_CLASS #define TCL_STORAGE_CLASS DLLEXPORT const RngStubs* rngStubsPtr; /*
*----------------------------------------------------------------------
*
* Rng_InitStubs --
*
* Checks that the correct version of Rng is loaded and that it
* supports stubs. It then initialises the stub table pointers.
*
* Results:
* The actual version of Rng that satisfies the request, or
* NULL to indicate that an error occurred.
*
* Side effects:
* Sets the stub table pointers.
*
*----------------------------------------------------------------------
*/ #ifdef Rng_InitStubs #undef Rng_InitStubs #endif char * Rng_InitStubs(Tcl_Interp *interp, CONST char *version, int exact) {
CONST char *actualVersion;
actualVersion = Tcl_PkgRequireEx(interp, "rng", version, exact, (ClientData *) &rngStubsPtr);
if (!actualVersion) { return NULL;
}
if (!rngStubsPtr) { Tcl_SetResult(interp, "This implementation of Rng does not support stubs", TCL_STATIC); return NULL;
}
return (char*) actualVersion; }
As the set of examples is a bit large, and growing, it has been put into a separate document. Please see section "Embedding C" in the document about Using CriTcl.
The latest changes are found at the top.
Extended critcl::cproc's argument handling to treat an argument args as variadic if it is the last argument of the procedure.
Further fixed the generated header to handle multiple inclusion.
Bumped version to 1.1.
Fixed the generated header to handle multiple inclusion.
Bumped version to 1.0.1.
Note that mode "compile & run" is not as blessed and still uses content-hasing with md5 to ensure unique file names in its per-user object cache.
As part of this the signature of command critcl::name2c changed. The command now delivers a list of four values instead of three. The new value was added at the end.
Further adapted the implementation of package critcl::class, a user of critcl::name2c. This package is now at version 1.0.6 and requires critcl 3.1.6
Lastly fixed the mis-handling of option -cname in critcl::ccommand, and critcl::cproc.
This package simplifies the creation of code associating data with an interpreter via Tcl's Tcl_(Get|Set)AssocData() APIs. The user can concentrate on his data while all the necessary boilerplate C code to support this is generated by the package.
This package uses several of the new features which were added to the core critcl package, see below.
This package simplifies the creation of C level objects with class and instance commands. The user can write a class definition with class- and instance-variables and -methods similar to a TclOO class, with all the necessary boilerplate C code to support this generated by the package.
This package uses several of the new features which were added to the core critcl package, see below.
The new packages critcl::iassoc and critcl::class (see above) are users of this command.
The new package critcl::class (see above) is a user of this command.
See the section Advanced: Diversions for details of the provided commands.
The new package critcl::class (see above) is a user of these facilities.
See the section Advanced: Location management for details of the provided commands.
The new packages critcl::iassoc and critcl::class (see above) are users of these facilities.
See the section Advanced: Extending cproc for details of the provided commands.
This new option, and -pkg, exclude each other. If both are specified the last used option takes precedence.
The generated package directory hierarchy is mostly self-contained, but not fully. It requires not only a working installation of Tcl, but also working installations of the packages md5 and cmdline. Both of these are provided by the Tcllib bundle. Not required, but recommended to have installed are any of the packages which can accelerate md5's operation, i.e. cryptkit, tcllibc, or Trf.
Note that this is a static scan. While the other build modes can (must) execute the ".critcl" file and make platform-specific decisions regarding the assembled C code, companion files, etc. the TEA wrap mode is not in a position to make platform-specific decisions. It has to wrap everything which might conceivably be needed when actually building. Hence the static scan. This has however its own set of problems, namely the inability to figure out any dynamic construction of companion file paths, at least on its own. Thus:
Please see section Stubs Table Management of the critcl package documentation for details.
Please see section Custom Build Configuration of the critcl package documentation for details.
Please see section Package Meta Data of the critcl package documentation for details.
Note that after using critcl::load / critcl::failed in a ".critcl" file it is not possible to use critcl commands in that file anymore. Doing so will throw an error.
Envisioned use is for the reporting of results determined by critcl::check and critcl::checklink during building, to help with debugging when something goes wrong with a check.
Jean Claude Wippler, Steve Landers, Andreas Kupries
This document, and the package it describes, will undoubtedly contain bugs and other problems. Please report them at https://github.com/andreas-kupries/critcl/issues. Ideas for enhancements you may have for either package, application, and/or the documentation are also very welcome and should be reported at https://github.com/andreas-kupries/critcl/issues as well.
C code, Embedded C Code, code generator, compile & run, compiler, dynamic code generation, dynamic compilation, generate package, linker, on demand compilation, on-the-fly compilation
Glueing/Embedded C code
Copyright (c) Jean-Claude Wippler Copyright (c) Steve Landers Copyright (c) 2011-2015 Andreas Kupries
3.1.17 | doc |