Bitbake log files

The Bitbake datastore has an API that allows you this access. Here is a list of available operations:. BitBake uses checksums or signatures along with the setscene to determine if a task needs to be run. This section describes the process. To help understand how BitBake does this, the section assumes an OpenEmbedded metadata-based example. This list is a place holder of content existed from previous work on the manual.

Some or all of it probably needs integrated into the subsections that make up this section. For now, I have just provided a short glossary-like description for each variable. Ultimately, this list goes away. BitBake's fetch module is a standalone piece of library code that deals with the intricacies of downloading source code and files from remote systems. Fetching source code is one of the corner stones of building software. As such, this module forms an important part of BitBake.

The current fetch module is called "fetch2" and refers to the fact that it is the second major version of the API. The original version is obsolete and removed from the codebase. Thus, in all cases, "fetch" refers to "fetch2" in this manual.

BitBake takes several steps when fetching source code or files. The fetcher codebase deals with two distinct processes in order: obtaining the files from somewhere cached or otherwise and then unpacking those files into a specific location and perhaps in a specific way. Getting and unpacking the files is often optionally followed by patching. Patching, however, is not covered by this module.

The code to execute the first part of this process, a fetch, looks something like the following:. This code sets up an instance of the fetch class. They variables can and are called with different variable names. In OpenEmbedded for example, the shared state sstate code uses the fetch module to fetch the sstate files.

When the download method is called, BitBake tries to fulfill the URLs by looking for source files in a specific search order:. Pre-mirror Sites: BitBake first uses pre-mirrors to try and find source files. Consider the following two URLs:.

In the former case, the URL is passed to the wget fetcher, which does not understand "git". Therefore, the latter case is the correct form since the Git fetcher does know how to use HTTP as a transport. Since network accesses are slow, Bitbake maintains a cache of files downloaded from the network. Any source files that are not local i. File integrity is of key importance for reproducing builds. For non-local archive downloads, the fetcher code can verify sha and md5 checksums to ensure the archives have been downloaded correctly.

If multiple URIs exist, you can specify the checksums either directly as in the previous example, or you can name the URLs. The following syntax shows how you name the URIs:. After a file has been downloaded and has had its checksum checked, a ".

BitBake uses this stamp during subsequent builds to avoid downloading or comparing a checksum for the file again. The unpack process usually immediately follows the download. A number of parameters exist that you can specify within the URL to govern the behavior of the unpack stage:. If set to "1", which is the default, the components are unpacked. If set to "0", the unpack stage leaves the file alone.

This parameter is useful when you want an archive to be copied in and not be unpacked. The unpack call automatically decompresses and extracts files with ". Z", ". As mentioned, the Git fetcher has its own unpack method that is optimized to work with Git trees. Basically, this method works by cloning the tree into the final directory. The process is completed using references so that there is only one central copy of the Git metadata needed.

Each submodule can support different URL parameters, which are described in the following sections. The filename you specify with in the URL can either be an absolute or relative path to a file. This submodule handles checking out files from the CVS version control system. You can configure it using a number of different variables:. This name is usually "cvs". A special value of "now" causes the checkout to be updated on every build.

As well as the standard username and password URL syntax, you can also configure the fetcher with various URL parameters:. By default, this protocol is "pserver". You can use "dir" for local directories.

You must supply this parameter. By default, the TAG is empty. The special value of "now" causes the checkout to be updated on every build. Effectively, you are renaming the output directory to which the module is unpacked. The tarball is expanded into the work directory. By default, the CVS metadata is removed. This fetcher obtains files from web and FTP servers. Internally, the fetcher uses the wget utility. The fetcher supports a parameter "downloadfilename" that allows the name of the downloaded file to be specified.

This fetcher submodule fetches code from the Subversion source control system. You must provide this parameter. You can think of this parameter as the top-level directory of the repository data you want. For "rsh", the "rsh" parameter is also used. Specific revisions are generally much safer to checkout rather than by date as they do not involve timezones e. By default, these directories are removed. This fetcher submodule fetches code from the Git source control system. This bare clone is then cloned into the work directory during the unpack stage when a specific tree is checked out.

This is done using alternates and by reference to minimize the amount of duplicate data on the disk and make the unpack process fast. The default is "git" when a hostname is set. If a hostname is not set, the Git protocol is "file". You can also use "http", "https", "ssh" and "rsync". Set this option for the URL where there is a custom routine to checkout code. The default is "0". You should set this parameter to "1" if revisions can become detached from branches.

In this case, the source mirror tarball is done per revision, which has a loss of efficiency. Rebasing the upstream Git repository could cause the current revision to disappear from the upstream repository. This option reminds the fetcher to preserve the local cache carefully for future use.

The default value for this parameter is "0". Set this option for the recipe that refers to the commit that is valid for a tag instead of the branch. Only the raw Git metadata is provided. This parameter implies the "nocheckout" parameter as well.

If unset, this is assumed to be "master". The number of branch parameters much match the number of name parameters. The default is "master". To correctly resolve tags, BitBake must access the network. For that reason, tags are often not used. As far as Git is concerned, the "tag" parameter behaves effectively the same as the "revision" parameter.

By default, the whole tree is checked out. No documentation currently exists for these lesser used fetcher submodules. However, you might find the code helpful and readable. This chapter lists common variables used by BitBake and gives an overview of their function and contents.

The variables listed in this glossary are specific to BitBake. Consequently, the descriptions are limited to that context. Also, variables exist in other systems that use BitBake e. The Yocto Project and OpenEmbedded that have names identical to those found in this glossary. For such cases, the variables in those systems extend the functionality of the variable as it is described here in this glossary.

Finally, there are variables mentioned in this glossary that do not appear in the BitBake glossary. These other variables are variables used in systems that use BitBake. Lists recipe names PN values BitBake does not attempt to build. Instead, BitBake assumes these recipes have already been built. An example is git-native , which when specified allows for the Git binary from the host to be used rather than building git-native.

The directory in which BitBake executes functions during a recipe's build process. Specifies the path to a log file into which BitBake's user interface writes output during the build. Contains the name of the currently running task. Defines how BitBake handles situations where an append file. This condition often occurs when layers get out of sync e. The default fatal behavior is safest because it is the sane reaction given something is out of sync. It is important to realize when your changes are no longer being applied.

The default task to use when none is specified e. Monitors disk space and available inodes during the build and allows you to control the build based on these parameters. Disk space monitoring is disabled by default.

When setting this variable, use the following form:. No disk monitoring occurs for the free inodes in this case. No disk space monitoring for the directory itself occurs in this case. During the build, subsequent warnings are issued each time disk space or number of free inodes further reduces by the respective interval. When specifying the variable in your configuration file, use the following form:.

Subsequent warnings based on the interval occur each time a respective interval is reached beyond the initial warning i. Specifies the internal whitelist of variables to allow through from the external environment into BitBake's datastore. Specifies an additional set of variables to allow through whitelist from the external environment into BitBake's datastore. Contains the filename of the recipe that owns the currently running task.

Anyone wishing to create a source mirror would want to enable this variable. For performance reasons, creating and placing tarballs of the Git repositories is not the default action by BitBake. Lists variables that are excluded from base configuration checksum, which is used to determine if the cache can be reused. One of the ways BitBake determines whether to re-parse the main metadata is through checksums of the variables in the datastore of the base configuration data.

There are variables that you typically want to exclude when checking whether or not to re-parse and thus rebuild the cache. If you did not exclude them, BitBake would never reuse the cache. Lists variables that are excluded from checksum and dependency data. Variables that are excluded can therefore change without affecting the checksum mechanism. A common example would be the variable for the path of the build. BitBake's output should not and usually does not depend on the directory in which it was built.

Specifies the name of the function to call during the "setscene" part of the task's execution in order to validate the list of task hashes. The function returns the list of setscene tasks that should be executed. At this point in the execution of the code, the objective is to quickly verify if a given setscene function is likely to work or not. It's easier to check the list of setscene functions in one pass than to call many individual tasks. The returned list need not be completely accurate.

A given setscene task can still later fail. However, the more accurate the data returned, the more efficient the build will be. Used in combination with the ConfigParsed event to trigger re-parsing the base metadata i.

The ConfigParsed event can set the variable to trigger the re-parse. You must be careful to avoid recursive loops with this functionality.

If you want to force log files to take a specific name, you can set this variable in a configuration file. Allows BitBake to run at a specific priority i. System permissions usually mean that BitBake can reduce its priority but not raise it again. Disables network access in the BitBake fetcher modules. With this access disabled, any command that attempts to access the network becomes an error. Disabling network access is useful for testing source mirrors, running builds when not connected to the Internet, and when operating in certain kinds of firewall environments.

The maximum number of tasks BitBake should run in parallel at any one time. If your host development system supports multiple cores, a good rule of thumb is to set this variable to twice the number of cores. Sets the number of threads BitBake uses when parsing. By default, the number of threads is equal to the number of cores on the system. Contains a copy of the original external environment in which BitBake was run.

The copy is taken before any whitelisted variable values are filtered into BitBake's datastore. Disables whitelisting and instead allows all variables through from the external environment into BitBake's datastore. Specifies the name of the executable script files i. If you want to force run files to take a specific name, you can set this variable in a configuration file. Contains the name of the currently executing task. Selects the name of the scheduler to use for the scheduling of BitBake tasks.

Three options exist:. Using this option causes tasks to be ordered numerically as they are parsed. The "speed" option is the default. Defines custom schedulers to import.

Custom schedulers need to be derived from the RunQueueScheduler class. Specifies a function BitBake calls that determines whether BitBake requires a setscene dependency to be met. When running a setscene task, BitBake needs to know which dependencies of that setscene task also need to be run. Whether dependencies also need to be run is highly dependent on the metadata. The function specified by this variable returns a "True" or "False" depending on whether the dependency needs to be met.

Specifies a function to call that verifies the list of planned task execution before the main task execution happens. The function is called once BitBake has a list of setscene tasks that have run and either succeeded or failed.

The function allows for a task list check to see if they make sense. Even if BitBake was planning to skip a task, the returned value of the function can force BitBake to run the task, which is necessary under certain metadata defined circumstances. Lists variable flags varflags that can be safely excluded from checksum and dependency data for keys in the datastore.

When generating checksum or dependency data for keys in the datastore, the flags set against that key are normally included in the checksum. For more information on varflags, see the " Variable Flags " section. Defines the name of the signature handler BitBake uses. The signature handler defines the way stamp files are created and handled, if and how the signature is incorporated into the stamps, and how the signature itself is generated.

A new signature handler can be added by injecting a class derived from the SignatureGenerator class into the global namespace.

Defines the behavior of the fetcher when it interacts with source control systems and dynamic source revisions. With this policy, there is no cache. The "clear" policy is the default.

Defines the mode used for how timestamps of stamp files are compared. You can set the variable to one of the following modes:. This is the default mode. Lists files whose stamp file timestamps are compared when the stamp policy mode is set to "whitelist". Sets a more strict checksum mechanism for non-local URLs. Setting this variable to a value causes BitBake to report an error if it encounters a non-local URL that does not have at least one checksum specified.

Allows specific tasks to change their priority i. You can use this variable in combination with task overrides to raise or lower priorities of specific tasks. For example, on the Yocto Project autobuilder, QEMU emulation in images is given a higher priority as compared to build tasks to ensure that images do not suffer timeouts on loaded systems. Within an executing task, this variable holds the hash of the task as returned by the currently enabled signature generator.

Controls how verbose BitBake is during builds. If set, shell scripts echo commands and shell script output appears on standard out stdout. Specifies if the current context is executing a task. BitBake sets this variable to "1" when a task is being executed.

The value is not set when the task is in server context during parsing or event handling. Allows you to extend a recipe so that it builds variants of the software.

To build a different variant of the recipe with a minimal amount of code, it usually is as simple as adding the variable to your recipe. Here are two examples. The "native" variants are from the OpenEmbedded Core metadata:. Sets the BitBake debug output level to a specific value as incremented by the -d command line option. Lists the names of configured layers. This variable is useful in situations where the same recipe appears in more than one layer.

Setting this variable allows you to prioritize a layer against other layers that contain the same recipe - effectively letting you control the precedence for the multiple layers. The precedence established through this variable stands regardless of a recipe's version PV variable. For example, the value 6 has a higher precedence than the value 5.

Contains a space-separated list of all of all files that BitBake's parser included during parsing of the current file. If set to a value, enables printing the task log when reporting a failed task. Lists the layers to enable during the build. This variable is defined in the bblayers. This example enables four layers, one of which is a custom, user-defined layer named meta-mykernel.

Prevents BitBake from processing recipes and recipe append files. BitBake ignores any recipe or recipe append files that match the expression. It is as if BitBake does not see them at all. Consequently, matching files are not parsed or otherwise used by BitBake.

The value you provide is passed to Python's regular expression compiler. The expression is compared against the full paths to the files. If you want to mask out multiple directories or recipes, use the vertical bar to separate the regular expression fragments. This next example masks out multiple directories and individual recipes:.

Used by BitBake to locate class. This variable is analogous to the PATH variable. Set the variable as you would any environment variable and then run BitBake:. Points to the server that runs memory-resident BitBake. The variable is only used when you employ memory-resident BitBake. Allows a single recipe to build multiple versions of a project from a single recipe file.

Used to specify the UI module to use when running BitBake. Using this variable is equivalent to using the -u command-line option. A name assigned to the build. The name defaults to a datetime stamp of when the build was started but can be defined by the metadata. Specifies the directory BitBake uses to store a cache of the metadata so it does not need to be parsed every time BitBake is started.

The most common usage of this is variable is to set it to "-1" within a recipe for a development version of a piece of software. Lists a recipe's build-time dependencies i. Consider this simple example for two recipes named "a" and "b" that produce similarly named packages. This means anything that recipe "b" puts into sysroot is available when recipe "a" is configuring itself. The central download directory used by the build process to store downloads.

Directs BitBake to exclude a recipe from world builds i. During world builds, BitBake locates, parses and builds all recipes found in every layer exposed in the bblayers. To exclude a recipe from a world build using this variable, set the variable to "1" in the recipe. Contains the command to use when running a shell script in a fakeroot environment.

See these entries in the glossary for more information. Contains the command that starts the bitbake-worker process in the fakeroot environment. Lists directories to create before running a task in the fakeroot environment. Lists environment variables to set when running a task in the fakeroot environment. Lists environment variables to set when running a task that is not in the fakeroot environment.

Defines the command the BitBake fetcher module executes when running fetch operations. You need to use an override suffix when you use the variable e.

Points at the current file. BitBake sets this variable during the parsing process to identify the file being parsed. BitBake also sets this variable when a recipe is being executed to identify the recipe file. Specifies directories BitBake uses when searching for patches and files. The variable behaves like a shell PATH environment variable. The value is a colon-separated list of directories that are searched left-to-right in order. Website where more information about the software the recipe is building can be found.

Causes the named class to be inherited at this point during parsing. The variable is only valid in configuration files. Lists the layers, separated by spaces, upon which this recipe depends. Optionally, you can specify a specific layer version for a dependency by adding it to the end of the layer name with a colon, e. BitBake produces an error if any dependency is missing or the version numbers do not match exactly if specified.

You must also use the specific layer name as a suffix to the variable e. When used inside the layer. This variable is not available outside of layer. Optionally specifies the version of a layer as a single number. Specifies additional paths from which BitBake gets source code. When the build system searches for source code, it first tries the local download directory. Allows you to suppress BitBake warnings caused when building two separate recipes that provide the same output.

Bitbake normally issues a warning when building two different recipes where each provides the same output. This scenario is usually something the user does not want. You can use this variable to suppress BitBake's warnings. To use the variable, list provider names e. You can find more information on how overrides are handled in the " Conditional Syntax Overrides " section. A promise that your recipe satisfies runtime dependencies for optional modules that are found in other recipes. The epoch of the recipe.

By default, this variable is unset. The variable is used to make upgrades possible when the versioning scheme changes in some backwards incompatible way. Specifies the directory BitBake uses to store data that should be preserved between builds. Specifies the recipe or package name and includes all version and revision numbers i. Determines which recipe should be given preference when multiple recipes provide the same item.

You should always suffix the variable with the name of the provided item, and you should set it to the PN of the recipe to which you want to give precedence. Some examples:. Determines which recipe should be given preference for cases where multiple recipes provide the same item. If there are multiple versions of recipes available, this variable determines which recipe should be given preference.

You must always suffix the variable with the PN you want to select, and you should set PV accordingly for precedence. Here are two examples:.

Typically, you would add a specific server for the build system to attempt before any others by adding something like the following to your configuration:.

A list of aliases that a recipe also provides. The network based PR service host and port. You must set the variable if you want to automatically start a local PR service. Lists a package's runtime dependencies i. If a package in this list cannot be found during the build, you will get a build error. For example, suppose you are building a development package that depends on the perl package.

In the example, the development package depends on the perl package. BitBake supports specifying versioned dependencies. Although the syntax varies depending on the packaging format, BitBake hides these differences from you. For operator , you can specify the following:. For example, the following sets up a dependency on version 1. A list of package name aliases that a package also provides. As with all package-controlling variables, you must always use the variable in conjunction with a package name override.

A list of packages that extends the usability of a package being built. The package being built does not depend on this list of packages in order to successfully build, but needs them for the extended usability. BitBake supports specifying versioned recommends. For example, the following sets up a recommend on version 1. The list of source files - local or remote. This variable tells BitBake which bits to pull for the build and how to pull them. The default action is to unpack the file.

This option is useful for unusual tarballs or other archives that do not have their files already in a subdirectory within the archive. The date of the source code used to build the package. The revision of the source code used to build the package.

This variable applies only when using Subversion, Git, Mercurial and Bazaar. If you want to build a fixed revision and you want to avoid performing a query on the remote repository every time BitBake parses your recipe, you should specify a SRCREV that is a full revision identifier and not just a tag. The system needs help constructing these values under these circumstances.

Consider an example with URLs named "machine" and "meta". And, this placeholder is placed at the start of the returned string. Specifies the base path used to create recipe stamp files. The path to an actual stamp file is constructed by evaluating this string and then appending additional information.

BitBake uses a clean operation to remove any other stamps it should be removing when creating a new stamp. Points to a directory were BitBake places temporary files, which consist mostly of task logs and scripts, when building a particular recipe. Points to the build directory. BitBake automatically sets this variable. The simplest example commonly used to demonstrate any new programming language or tool is the " Hello World " example.

This appendix demonstrates, in tutorial form, Hello World within the context of BitBake. The tutorial describes how to create a new project and the applicable metadata files necessary to allow BitBake to build it. Once you have the source code on your machine, the BitBake directory appears as follows:. At this point, you should have BitBake cloned to a directory that matches the previous listing except for dates and user names.

First, you need to be sure that you can run BitBake. Set your working directory to where your local BitBake files are and run the following command:. The recommended method to run BitBake is from a directory of your choice. To be able to run BitBake from any directory, you need to add the executable binary to your binary to your shell's environment PATH variable. First, look at your current PATH variable by entering the following:.

You should now be able to enter the bitbake command from the command line while working from any directory. The overall goal of this exercise is to build a complete "Hello World" example utilizing task and layer concepts. The BitBake recipes consist of the source URL http, https, ftp, cvs, svn, git, local file system of the package, dependencies and compile or install options.

During the build process they are used to track dependencies, performing native or cross-compilation of the package and package it so that it is suitable for installation on the local or a target device.

It is also possible to create complete images consisting of a root file system and kernel. As a first step in a cross-build setup, the framework will attempt to create a cross-compiler toolchain suited for the target platform.

Log files with same content after build [closed] Ask Question. Asked 4 years ago. Active 4 years ago. Viewed times. Brendan 2 2 gold badges 16 16 silver badges 29 29 bronze badges. You can join yocto channel on freenode if you have other questions — Nayfe. Add a comment. Active Oldest Votes. Nayfe Nayfe 1, 11 11 silver badges 18 18 bronze badges.