DOKK / manpages / debian 11 / kas / kas.1.en
KAS(1) kas KAS(1)

kas - kas Documentation

This tool provides an easy mechanism to setup bitbake based projects.

The OpenEmbedded tooling support starts at step 2 with bitbake. The downloading of sources and then configuration has to be done by hand. Usually, this is explained in a README. Instead kas is using a project configuration file and does the download and configuration phase.

Key features provided by the build tool:

  • clone and checkout bitbake layers
  • create default bitbake settings (machine, arch, ...)
  • launch minimal build environment, reducing risk of host contamination
  • initiate bitbake build process

This project depends on

  • Python 3
  • distro Python 3 package
  • jsonschema Python 3 package
  • PyYAML Python 3 package (optional, for yaml file support)

To install kas into your python site-package repository, run:

$ sudo pip3 install .


There are (at least) four options for using kas:

  • Install it locally via pip to get the kas command.
  • Use the container image locally. In this case, download the kas-container script from the kas repository and use it in place of the kas command. The script version corresponds to the kas tool and the kas image version.
  • Use the container image in CI. Specify ghcr.io/siemens/kas/kas[-isar][:<x.y>] in your CI script that requests a container image as runtime environment. See https://github.com/orgs/siemens/packages/container/kas%2Fkas/31765 and https://github.com/orgs/siemens/packages/container/kas%2Fkas-isar/31794 for all available images.
  • Use the run-kas wrapper from this directory. In this case, replace kas in the examples below with path/to/run-kas.

Start build:

$ kas build /path/to/kas-project.yml


Alternatively, experienced bitbake users can invoke usual bitbake steps manually, e.g.:

$ kas shell /path/to/kas-project.yml -c 'bitbake dosfsutils-native'


kas will place downloads and build artifacts under the current directory when being invoked. You can specify a different location via the environment variable KAS_WORK_DIR.

1.
Initial build/setup:

$ mkdir $PROJECT_DIR
$ cd $PROJECT_DIR
$ git clone $PROJECT_URL meta-project
$ kas build meta-project/kas-project.yml


2.
Update/rebuild:

$ cd $PROJECT_DIR/meta-project
$ git pull
$ kas build kas-project.yml



kas sub-commands are implemented by a series of plugins. Each plugin typically provides a single command.

build plugin

This plugin implements the kas build command.

When this command is executed, kas will checkout repositories, setup the build environment and then invoke bitbake to build the targets selected in the chosen config file.

For example, to build the configuration described in the file kas-project.yml you could run:

kas build kas-project.yml


checkout plugin

This plugin implements the kas checkout command.

When this command is executed, kas will checkout repositories and set up the build directory as specified in the chosen config file. This command is useful if you need to inspect the configuration or modify any of the checked out layers before starting a build.

For example, to setup the configuration described in the file kas-project.yml you could run:

kas checkout kas-project.yml


for-all-repos plugin

This plugin implements the kas for-all-repos command.

When this command is executed, kas will checkout the repositories listed in the chosen config file and then execute a specified command in each repository. It can be used to query the repository status, automate actions such as archiving the layers used in a build or to execute any other required commands.

For example, to print the commit hashes used by each repository used in the file kas-project.yml (assuming they are all git repositories) you could run:

kas for-all-repos kas-project.yml 'git rev-parse HEAD'


The environment for executing the command in each repository is extended to include the following variables:

  • KAS_REPO_NAME: The name of the current repository determined by either the name property or by the key used for this repo in the config file.
  • KAS_REPO_PATH: The path of the local directory where this repository is checked out, relative to the directory where kas is executed.
  • KAS_REPO_URL: The URL from which this repository was cloned, or an empty string if no remote URL was given in the config file.
  • KAS_REPO_REFSPEC: The refspec which was checked out for this repository, or an empty string if no refspec was given in the config file.



shell plugin

This plugin implements the kas shell command.

When this command is executed, kas will checkout repositories, setup the build environment and then start a shell in the build environment. This can be used to manually run bitbake with custom command line options or to execute other commands such as runqemu.

For example, to start a shell in the build environment for the file kas-project.yml you could run:

kas shell kas-project.yml


Or to invoke qemu to test an image which has been built:

kas shell kas-project.yml -c 'runqemu'


Currently, JSON and YAML are supported as the base file formats. Since YAML is arguably easier to read, this documentation focuses on the YAML format.

# Every file needs to contain a header, that provides kas with information
# about the context of this file.
header:


  # The `version` entry in the header describes for which configuration


  # format version this file was created for. It is used by kas to figure


  # out if it is compatible with this file. The version is an integer that


  # is increased on every format change.


  version: x
# The machine as it is written into the `local.conf` of bitbake.
machine: qemux86-64
# The distro name as it is written into the `local.conf` of bitbake.
distro: poky
repos:


  # This entry includes the repository where the config file is located


  # to the bblayers.conf:


  meta-custom:


  # Here we include a list of layers from the poky repository to the


  # bblayers.conf:


  poky:


    url: "https://git.yoctoproject.org/git/poky"


    refspec: 89e6c98d92887913cadf06b2adb97f26cde4849b


    layers:


      meta:


      meta-poky:


      meta-yocto-bsp:


A minimal input file consists out of the header, machine, distro, and repos.

Additionally, you can add bblayers_conf_header and local_conf_header which are strings that are added to the head of the respective files (bblayers.conf or local.conf):

bblayers_conf_header:


  meta-custom: |


    POKY_BBLAYERS_CONF_VERSION = "2"


    BBPATH = "${TOPDIR}"


    BBFILES ?= ""
local_conf_header:


  meta-custom: |


    PATCHRESOLVE = "noop"


    CONF_VERSION = "1"


    IMAGE_FSTYPES = "tar"


meta-custom in these examples should be a unique name (in project scope) for this configuration entries. We assume that your configuration file is part of a meta-custom repository/layer. This way its possible to overwrite or append entries in files that include this configuration by naming an entry the same (overwriting) or using an unused name (appending).

It's currently possible to include kas configuration files from the same repository/layer like this:

header:


  version: x


  includes:


    - base.yml


    - bsp.yml


    - product.yml


The specified files are addressed relative to your current configuration file.

It's also possible to include configuration files from other repos like this:

header:


  version: x


  includes:


    - repo: poky


      file: kas-poky.yml


    - repo: meta-bsp-collection


      file: hw1/kas-hw-bsp1.yml


    - repo: meta-custom


      file: products/product.yml
repos:


  meta-custom:


  meta-bsp-collection:


    url: "https://www.example.com/git/meta-bsp-collection"


    refspec: 3f786850e387550fdab836ed7e6dc881de23001b


    layers:


      # Additional to the layers that are added from this repository


      # in the hw1/kas-hw-bsp1.yml, we add here an additional bsp


      # meta layer:


      meta-custom-bsp:


  poky:


    url: "https://git.yoctoproject.org/git/poky"


    refspec: 89e6c98d92887913cadf06b2adb97f26cde4849b


    layers:


      # If `kas-poky.yml` adds the `meta-yocto-bsp` layer and we


      # do not want it in our bblayers for this project, we can


      # overwrite it by setting:


      meta-yocto-bsp: exclude


The files are addressed relative to the git repository path.

The include mechanism collects and merges the content from top to bottom and depth first. That means that settings in one include file are overwritten by settings in a latter include file and entries from the last include file can be overwritten by the current file. While merging all the dictionaries are merged recursively while preserving the order in which the entries are added to the dictionary. This means that local_conf_header entries are added to the local.conf file in the same order in which they are defined in the different include files. Note that the order of the configuration file entries is not preserved within one include file, because the parser creates normal unordered dictionaries.

When specifying the kas configuration file on the command line, additional configurations can be included ad-hoc:

$ kas build kas-base.yml:debug-image.yml:board.yml


This is equivalent to static inclusion from some kas-combined.yml like this:

header:


  version: x


  includes:


    - kas-base.yml


    - debug.image.yml


    - board.yml


Command line inclusion allows to create configurations on-demand, without the need to write a kas configuration file for each possible combination.

Note that all configuration files combined via the command line either have to come from the same repository or have to live outside of any versioning control. kas will refuse any other combination in order to avoid complications and configuration flaws that can easily emerge from them.

The header of every kas configuration file. It contains information about the context of the file.
version: integer [required]
Lets kas check if it is compatible with this file. See the configuration format changelog for the format history and the latest available version.
A list of configuration files this current file is based on. They are merged in order they are stated. So a latter one could overwrite settings from previous files. The current file can overwrite settings from every included file. An item in this list can have one of two types:
The path to a kas configuration file, relative to the current file.
If files from other repositories should be included, choose this representation.
The id of the repository where the file is located. The repo needs to be defined in the repos dictionary as <repo-id>.
The path to the file relative to the root of the repository.



Defines the bitbake-based build system. Known build systems are openembedded (or oe) and isar. If set, this restricts the search of kas for the init script in the configured repositories to oe-init-build-env or isar-init-build-env, respectively. If kas-container finds this property in the top-level kas configuration file (includes are not evaluated), it will automatically select the required container image and invocation mode.
This key can be used to set default values for various properties. This may help you to avoid repeating the same property assignment in multiple places if, for example, you wish to use the same refspec for all repositories.
This key can contain default values for some repository properties. If a default value is set for a repository property it may still be overridden by setting the same property to a different value in a given repository.
Sets the default refspec property applied to all repositories that do not override this.
This key can contain default values for some repository patch properties. If a default value is set for a patch property it may still be overridden by setting the same property to a different value in a given patch.
Sets the default repo property applied to all repository patches that do not override this.



Contains the value of the MACHINE variable that is written into the local.conf. Can be overwritten by the KAS_MACHINE environment variable and defaults to qemux86-64.
Contains the value of the DISTRO variable that is written into the local.conf. Can be overwritten by the KAS_DISTRO environment variable and defaults to poky.
target: string [optional] or list [optional]
Contains the target or a list of targets to build by bitbake. Can be overwritten by the KAS_TARGET environment variable and defaults to core-image-minimal. Space is used as a delimiter if multiple targets should be specified via the environment variable.
Contains environment variable names with the default values. These variables are made available to bitbake via BB_ENV_EXTRAWHITE and can be overwritten by the variables of the environment in which kas is started.
Contains the task to build by bitbake. Can be overwritten by the KAS_TASK environment variable and defaults to build.
Contains the definitions of all available repos and layers.
<repo-id>: dict [optional]
Contains the definition of a repository and the layers, that should be part of the build. If the value is None, the repository, where the current configuration file is located is defined as <repo-id> and added as a layer to the build.
Defines under which name the repository is stored. If its missing the <repo-id> will be used.
The url of the repository. If this is missing, no version control operations are performed.
The type of version control repository. The default value is git and hg is also supported.
The refspec that should be used. If url was specified but no refspec the revision you get depends on the defaults of the version control system used.
The path where the repository is stored. If the url and path is missing, the repository where the current configuration file is located is defined. If the url is missing and the path defined, this entry references the directory the path points to. If the url as well as the path is defined, the path is used to overwrite the checkout directory, that defaults to kas_work_dir + repo.name. In case of a relative path name kas_work_dir is prepended.
Contains the layers from this repository that should be added to the bblayers.conf. If this is missing or None or and empty dictionary, the path to the repo itself is added as a layer.
<layer-path>: enum [optional]
Adds the layer with <layer-path> that is relative to the repository root directory, to the bblayers.conf if the value of this entry is not in this list: ['disabled', 'excluded', 'n', 'no', '0', 'false']. This way it is possible to overwrite the inclusion of a layer in latter loaded configuration files.

Contains the patches that should be applied to this repo before it is used.
<patches-id>: dict [optional]
One entry in patches with its specific and unique id. All available patch entries are applied in the order of their sorted <patches-id>.
The identifier of the repo where the path of this entry is relative to.
The path to one patch file or a quilt formatted patchset directory.




This contains strings that should be added to the bblayers.conf before any layers are included.
<bblayers-conf-id>: string [optional]
A string that is added to the bblayers.conf. The entry id (<bblayers-conf-id>) should be unique if lines should be added and can be the same from another included file, if this entry should be overwritten. The lines are added to bblayers.conf in the same order as they are included from the different configuration files.

This contains strings that should be added to the local.conf.
<local-conf-id>: string [optional]
A string that is added to the local.conf. It operates in the same way as the bblayers_conf_header entry.

Defines the proxy configuration bitbake should use. Every entry can be overwritten by the respective environment variables.

  • http_proxy: string [optional]
  • https_proxy: string [optional]
  • no_proxy: string [optional]



kas - setup tool for bitbake based project

usage: kas [-h] [--version] [-d] {build,checkout,for-all-repos,shell} ...


Positional Arguments

Possible choices: build, checkout, for-all-repos, shell

sub command help


Named Arguments

show program's version number and exit
Enable debug logging

Default: False


Checks out all necessary repositories and builds using bitbake as specified in the configuration file.

kas build [-h] [--skip SKIP] [--force-checkout] [--update] [--target TARGET]


          [-c TASK]


          config [extra_bitbake_args ...]


Positional Arguments

Config file
Extra arguments to pass to bitbake

Named Arguments

Skip build steps

Default: []

Always checkout the desired refspec of each repository, discarding any local changes

Default: False

Pull new upstream changes to the desired refspec even if it is already checked out locally

Default: False

Select target to build
Select which task should be executed

Checks out all necessary repositories and sets up the build directory as specified in the configuration file.

kas checkout [-h] [--skip SKIP] [--force-checkout] [--update] config


Positional Arguments

Config file

Named Arguments

Skip build steps

Default: []

Always checkout the desired refspec of each repository, discarding any local changes

Default: False

Pull new upstream changes to the desired refspec even if it is already checked out locally

Default: False


Runs a specified command in all checked out repositories.

kas for-all-repos [-h] [--skip SKIP] [--force-checkout] [--update]


                  config command


Positional Arguments

Config file
Command to be executed as a string.

Named Arguments

Skip build steps

Default: []

Always checkout the desired refspec of each repository, discarding any local changes

Default: False

Pull new upstream changes to the desired refspec even if it is already checked out locally

Default: False


Run a shell in the build environment.

kas shell [-h] [--skip SKIP] [--force-checkout] [--update] [-k] [-c COMMAND]


          config


Positional Arguments

Config file

Named Arguments

Skip build steps

Default: []

Always checkout the desired refspec of each repository, discarding any local changes

Default: False

Pull new upstream changes to the desired refspec even if it is already checked out locally

Default: False

Skip steps that change the configuration

Default: False

Run command

Default: ""


Environment variables Description
KAS_WORK_DIR The path of the kas work directory, current work directory is the default.
KAS_REPO_REF_DIR The path to the repository reference directory. Repositories in this directory are used as references when cloning. In order for kas to find those repositories, they have to be named in a specific way. The repo URLs are translated like this: "https://github.com/siemens/meta-iot2000.git" resolves to the name "github.com.siemens.meta-iot2000.git".
KAS_DISTRO KAS_MACHINE KAS_TARGET KAS_TASK This overwrites the respective setting in the configuration file.
KAS_PREMIRRORS Specifies alternatives for repo URLs. Just like bitbake PREMIRRORS, this variable consists of new-line separated entries. Each entry defines a regular expression to match a URL and, space- separated, its replacement. E.g.: "https://.*.somehost.io/ https://localmirror.net/"
SSH_PRIVATE_KEY Path to the private key file that should be added to an internal ssh-agent. This key cannot be password protected. This setting is useful for CI build servers. On desktop machines, an ssh-agent running outside the kas environment is more useful.
SSH_AUTH_SOCK SSH authentication socket. Used for cloning over SSH (alternative to SSH_PRIVATE_KEY).
DL_DIR SSTATE_DIR TMPDIR Environment variables that are transferred to the bitbake environment.
http_proxy https_proxy ftp_proxy no_proxy This overwrites the proxy configuration in the configuration file.
GIT_PROXY_COMMAND NO_PROXY Set proxy for native git fetches. NO_PROXY is evaluated by OpenEmbedded's oe-git-proxy script.
SHELL The shell to start when using the shell plugin.
TERM The terminal options used in the shell plugin.
AWS_CONFIG_FILE AWS_SHARED_CREDENTIALS_FILE Path to the awscli configuration and credentials file that are copied to the kas home dir.

This project uses pip to manage the package. If you want to work on the project yourself you can create the necessary links via:

$ pip3 install --user -e .


That will install a backlink ~/.local/bin/kas to this project. Now you are able to call it from anywhere.

Just run:

$ docker build -t <image_name> .


When you need a proxy to access the internet, add:

--build-arg http_proxy=<http_proxy> --build-arg https_proxy=<https_proxy> --build-arg ftp_proxy=<ftp_proxy> --build-arg no_proxy=<no_proxy>


to the call.

Project home:

•
https://github.com/siemens/kas



Source code:

  • https://github.com/siemens/kas.git
  • git@github.com:siemens/kas.git



Documentation:

•
https://kas.readthedocs.org



Mailing list:

  • kas-devel@googlegroups.com
  • Subscription:
  • kas-devel+subscribe@googlegroups.com
  • https://groups.google.com/forum/#!forum/kas-devel/join

•
Archives
  • https://groups.google.com/forum/#!forum/kas-devel
  • https://www.mail-archive.com/kas-devel@googlegroups.com/




kas.kas Module

This module is the main entry point for kas, setup tool for bitbake based projects

Setup the logging environment

Ignore SIGINT/SIGTERM in kas, let them be handled by our sub-processes

The actual main entry point of kas.

Creates an argparser for kas with all plugins.

The main function that operates as a wrapper around kas.

kas.libkas Module

This module contains the core implementation of kas.

Handles the log output of executed applications
This method is called when a line is received over stderr.

This method is called when a line is received over stdout.


Find a file within the paths array and returns its path.

Creates the build environment variables.

Applies the patches to the repositories.

Fetches the list of repositories to the kas_work_dir.

Runs a command synchronously.

Run a command asynchronously.

Adds an ssh key to the ssh-agent

Removes the identities and stops the ssh-agent instance

Disables ssh host key check

Starts the ssh-agent

kas.libcmds Module

This module contains common commands used by kas plugins.

Removes all the identities and stops the ssh-agent instance.
This method executes the command.


An abstract class that defines the interface of a command.
This method executes the command.


Finalizes the repo setup loop
TODO refactor protected-access


Prepares setting up repos including the include logic
This method executes the command.


A class that defines a set of commands as a loop.
Appends a command to the loop.

Executes the loop.


Contains commands and provides method to run them.
Appends commands to the command list.

Runs a command from the command list with respect to the configuration.


Applies the patches defined in the configuration to the repositories.
This method executes the command.


Ensures that the right revision of each repo is checked out.
This method executes the command.


Fetches repositories defined in the configuration
This method executes the command.


Creates the build directory.
This method executes the command.


Sets up the kas environment.
This method executes the command.


Sets up the home directory of kas.
This method executes the command.


Single step of the checkout repos loop
TODO refactor protected-access


Sets up the ssh agent configuration.
This method executes the command.


Writes bitbake configuration files into the build directory.
This method executes the command.


kas.config Module

This module contains the implementation of the kas configuration.

Implements the kas configuration based on config files.
Returns repos that are in config but not on disk

Returns the bblayers.conf header

Returns a list of bitbake targets

Returns the bitbake task

Returns the pre-selected build system

Returns the distro

Returns the configured environment variables from the configuration file with possible overwritten values from the environment.

Returns the local.conf header

Returns the machine

Returns the multiconfig array as bitbake string

Returns the list of repos.


kas.repos Module

This module contains the Repo class.

Provides the git functionality for a Repo.

Provides the hg functionality for a Repo.

Represents a repository in the kas configuration.
Returns a Repo instance depending on params.

Checks if path is under version control and returns its root path.


Provides a generic implementation for a Repo.
Applies patches to a repository asynchronously.

Checks out the correct revision of the repo.

Starts asynchronous repository fetch.


kas.includehandler Module

This module implements how includes of configuration files are handled in kas.

Class for exceptions that appear in the include mechanism.

Implements a handler where every configuration file should contain a dictionary as the base type with and 'includes' key containing a list of includes.

The includes can be specified in two ways: as a string containing the relative path from the current file or as a dictionary. The dictionary should have a 'file' key containing the relative path to the include file and optionally a 'repo' key containing the key of the repository. If the 'repo' key is missing the value of the 'file' key, it is treated the same as if just a string was defined, meaning the path is relative to the current config file. Otherwise it is interpreted relative to the repository path.

The includes are read and merged from the deepest level upwards.

repos -- A dictionary that maps repo names to directory paths
(config, repos)
config -- A dictionary containing the configuration repos -- A list of missing repo names that are needed to create a complete configuration




Class for exceptions that appear while loading a configuration file.

Load the configuration file and test if version is supported.

kas.plugins Module

This module contains and manages kas plugins

Get a list of all loaded kas plugin classes

Lookup a kas plugin class by name

Import all kas plugins

Register all kas plugins found in a module

  • Include mechanism
  • Version check

•
Configuration file versions are now integers

•
Including files from repos that are not defined in the current file

•
Task key that allows to specify which task to run (bitbake -c)

•
Target key now allows to be a list of target names

•
Using multiconfig:* targets adds appropriate BBMULTICONFIG entries to the local.conf automatically.

•
env key now allows to pass custom environment variables to the bitbake build process.

•
type property to repos to be able to express which version control system to use.

•
patches property to repos to be able to apply additional patches to the repo.

•
defaults key can now be used to set a default value for the repository property refspec and the repository patch property repo. These default values will be used if the appropriate properties are not defined for a given repository or patch.

•
build_system property to pre-select OE or Isar.

  • genindex
  • modindex
  • search

Daniel Wagner, Jan Kiszka, Claudius Heine

Siemens AG, 2017-2018

March 14, 2021 2.3.3