In this post we shall went through most steps needed to build up Linux images for Raspberry Pi 3B. It is recommended to have a 8GB or larger micro SD card, Raspberry Pi 3B or 3B+, and other peripherals to test the generated kernel image. This tutorial makes part of the Introduction to IIO driver development series of tutorials.

Get a Linux Tree

There are several repositories that contain the source code of the Linux kernel. These repositories are know as trees. Therefore, a Linux tree is a repo where some development for the kernel happens. A Linux tree may be widely known and downloaded (upstream), like Linus’ tree, or may be one that send its contributions (patches) to upstream trees.

Some examples of Linux trees are:

To get a Linux tree repo to start working with, you might use git to download (clone) it.

For instance, to get the IIO tree one might type:

git clone git://git.kernel.org/pub/scm/linux/kernel/git/jic23/iio.git

For the Raspberry Pi tree:

git clone git@github.com:raspberrypi/linux.git

For the Analog Devices tree:

git clone https://github.com/analogdevicesinc/linux.git

Throughout this series of tutorials, I’ll be using the Analog Devices tree. However, the steps taken here will be much similar to steps needed for working on other Linux trees.

More about Linux trees may be seen on some good articles like “The linux-next and -stable trees” from the LWN.net journal, and/or in great talks like Greg’s “Linux Kernel Development” at Git Merge.

A list of upstream Linux trees can be seen at https://git.kernel.org/.

Configure Compilation Symbols

The Kernel Build System (kbuild) uses the configurations symbols and respective values stored inside a file called “.config”. It is not recommended to edit this file manually unless you know exactly what you’re doing. The configuration options are sensible to ordering. Every entry may have its own dependencies. Disabling an option may limit the visibility of dependent entries. However, if you are wiling to play with .config files, you may specify a different config file setting with the KCONFIG_CONFIG variable, like:

export KCONFIG_CONFIG=.my_config

It may also be set up just when asking kbuild to compile the kernel:

make KCONFIG_CONFIG=.my_config

The kernel configuration symbols are defined in Kconfig files. Most directories inside the kernel source tree have a Kconfig file that includes (source) Kconfig files from its subdirectories. For instance, to add a configuration symbol for the AD7292 driver, the following entry would be added to the Kconfig file at drivers/iio/adc/Kconfig.

config AD7292
	tristate "Analog Devices AD7292 ADC driver"
	depends on SPI
	help
	  Say yes here to build support for Analog Devices AD7292
	  8 Channel ADC with temperature sensor.

	  To compile this driver as a module, choose M here: the
	  module will be called ad7292.

The config keyword defines a new symbol which in turn is presented as a configuration entry in the .config file, within tools like menuconfig and nconfig, or at compilation time. In particular, the AD7292 config symbol has the following attributes:

  • tristate: The type for the config option. It tells that this symbol stands for something that may be compiled to be included as a module (m), built in compiled (y), or not compiled at all (n). The type definition also accepts an optional input prompt which is displayed in the config item within tools like menuconfig or nconfig.
  • depends on: tells which other config entries this symbol depends on. If its dependencies are not satisfied, this symbol may become non-visible at configuration and/or compilation time. As an experiment, try to disable SPI support at Device Drivers. Note that, as you do so, AD7292 won’t be listed at Device Drivers → Industrial I/O support → Analog to digital converters anymore.
  • help: defines a help text to be displayed as auxiliary info.

There are a few ways to prepare a kernel compilation file. It may be done manually, with the aid of tools like menuconfig or nconfig, with a defconfig file. Defconfig files are intended to store only specific non-default values (i.e options we changed for our board or project) for compilation symbols. Some defconfig files for the arm architecture may be found at arch/arm/configs/. Commonly configurations used for Rasbperry Pi boards are stored in the files bcm2709_defconfig, bcm2835_defconfig, and bcmrpi_defconfig. Analog Devices also has a custom defconfig file at their repository (adi_bcm2709_defconfig). To add a custom compilation value for the AD7292 symbol, we would add the following line to adi_bcm2709_defconfig file:

CONFIG_AD7292=y

Comment: bcm comes from Broadcom, manufacturer of the processors used largely on Raspberry Pi boards. See the Raspberry Pi hardware documentation.

We may apply the configurations defined at a defconfig file passing its name as a make rule. Like this:

make adi_bcm2709_defconfig

This will set the values stored in adi_bcm2709_defconfig into the .config file to be used during kernel compilation.

Before compiling the kernel itself, there is one more thing that would be done. Although we have included a new configuration symbol for kernel compilation, we did not bound it to the compilation of the AD7292 driver. The kernel image is build by the kbuild Makefiles. Like the Kconfig files, kbuild Makefiles are present in most kernel directories. From Javier Canillas’ article at the Linux Journal:

“The whole build is done recursively — a top Makefile descends into its subdirectories and executes each subdirectory’s Makefile to generate the binary objects for the files in that directory. Then, these objects are used to generate the modules and the Linux kernel image”.

To have a C file compiled, a build goal has to be added to the Makefile in its directory. For instance, if we wanted our AD7292 driver to be compiled, we would add the following to drivers/iio/adc/Makefile:

obj-$(CONFIG_AD7292) += ad7292.o

Canillas also made a summary of the steps needed to add new kernel functionality:

To add a feature to a Linux kernel (such as the coin driver), you need to do three things:

  • Put the source file(s) in a place that makes sense, such as drivers/net/wireless for Wi-Fi devices or fs for a new filesystem.

  • Update the Kconfig for the subdirectory (or subdirectories) where you put the files with config symbols that allow you to choose to include the feature.

  • Update the Makefile for the subdirectory where you put the files, so the build system can compile your code conditionally.

More about attributes of kconfig symbols, kernel Makefiles, the kbuild system as a whole, can be found at the official documentation for the Kernel Build System. Also available inside the kernel source tree at: Documentation/kbuild/.

A nice thread regarding defconfig usage and its motivation can be found at Stack Overflow.

Instructions on how to configure and build the official Raspberry Pi tree can be found at the Raspberry Pi documentation: Configuring the kernel and Kernel building.

Javier Canillas’ great article explaining the whole kernel compilation process can be found at the Linux Journal.

Kernel Cross Compilation

Different processor architectures usually have different instruction sets and register names. Due to this, using a compiler that produces code for computers which processor belongs to the x86 architecture family (assuming you are programming on such a machine), will generate binaries that won’t work on arm processors. Because of this, we need to use a cross compiler. The cross compiler will use a different compilations process to generate binaries (one that can produce binaries compatible with the arm architecture).

Surely, there are several cross compilers to arm architecture on the Internet however, I’ll be using Linaro’s one (which is a customized GCC). By the time I am writing this tutorial, one of the latest Linaro cross compiling tools is Linaro GCC 7.

After downloaded, you may extract it with tar:

tar -xf gcc-linaro-7.4.1-2019.02-x86_64_arm-linux-gnueabi.tar.xz

To indicate that we want to use a cross compiler, we may set the CROSS_COMPILE variable to point to the Linaro’s GCC. For instance, if you have extracted it one directory above the kernel source tree, you might set:

export CROSS_COMPILE=../gcc-linaro-7.4.1-2019.02-x86_64_arm-linux-gnueabi/bin/arm-linux-gnueabi-

As stated in the documentation:

By default, the top Makefile sets $(ARCH) to be the same as the host system architecture. For a cross build, a user may override the value of $(ARCH) on the command line: make ARCH=m68k …

So, we need to specify the target architecture we want binaries be generated for. This is done by setting the ARCH variable. In our case:

export ARCH=arm

Note: All these variables can be set just before compilation, on the command line:

make ARCH=arm export CROSS_COMPILE=../gcc-linaro-7.4.1-2019.02-x86_64_arm-linux-gnueabi/bin/arm-linux-gnueabi-

That’s a pretty long command in my opinion. This is the reason why I prefer to export the values for the variables in separate commands, instead of preparing a giant one that may be not as much readable. Though, each one decides what works best for themselves, from now on, I will assume the ARCH and CROSS_COMPILE variables are exported.

Now, when kbuild start compiling the Linux kernel files, it will use the cross compiler set to do so.

Finally, we are ready to compile the Linux kernel to be use in our Raspberry Pi. =D

Type:

make zImage modules dtbs -j4

Cross your fingers, press Enter and go fetch a mug of coffee or any drink you like. The compilation process might take some time (around half an hour in my quad core machine). If you’ve got more than 4 cores in your machine, you may put them to work increasing the number of threads used by the kbuild. This is done by setting the j flag. For instance, if you want the compilation process to be carried on with 8 threads, you may use -j8. The dtbs rule makes kbuild to compile the device tree files at arch/arm/boot/dts into device tree blobs to be used during system boot. The modules rule indicates to build code that is marked as modules (m) (duhh). The zImage refers to architecture specific kind of image we want to be generated. Different form the default vmlinux image, the zImage is a compressed kernel image.

If everything goes well, the last output lines should look like this:

  LD      arch/arm/boot/compressed/vmlinux
  OBJCOPY arch/arm/boot/zImage
  Kernel: arch/arm/boot/zImage is ready

The compilation will create (or overwrite) the following files:

  • arch/arm/boot/zImage: the kernel image
  • modules.order, modules.builtin, and modules.builtin.modinfo: holding information about which modules are built into the kernel.
  • arch/arm/boot/dts/*.dtb: device tree blob files. These files describe board hardware in a compact way.
  • arch/arm/boot/dts/overlays/*.dtbo: device tree overlay blobs. These are used to extend device tree hardware description.

That’s it. Congratulations, you’ve compiled the Linux kernel for Raspberry Pi! =D

If you got any trouble through this process, you might find additional references and tips at:

There’s also a nice introductory cross compiling video class from Mateus Castello, if you can understand portuguese.

Now I’ll go through the steps for testing the compiled kernel.

Install an Operating System: Burn a Raspbian SD card

To properly test our newly compiled kernel image, I recommend to prepare a micro SD card with Raspbian. Also, once Graphical User Interface (GUI) will not be needed, I would recommend to use the Raspbian Lite version.

The official installation guide is very intuitive and comprehensive. Therefore, I will give no further explanation about it.

Prepare The SD Card

Once you have flashed a Raspbian image to a SD card there should be two partitions within it: boot and rootfs. Some OS automatically mount the partitions present on SD cards when they are inserted. You may use those mount points to copy files in and out of the card. If you got no mount points, the device and partitions to be mounted may be found with fdisk:

sudo fdisk -l

The boot partition is around 50 megabytes size and is FAT32 formatted (at /dev/mmcblk0p1 in my case). It contains device tree blob files, overlays, the Linux kernel image, and configuration files needed to boot the system. The rootfs partition uses the rest of the SD card storage and has a Linux compatible file system (at /dev/mmcblk0p2). It contains the filesystem for the Raspbian operating system.

You may mount these partition to be able to copy the files. For instance:

sudo mkdir /mnt/rpi-boot
sudo mkdir /mnt/rpi-rootfs
sudo mount /dev/mmcblk0p1 /mnt/rpi-boot
sudo mount /dev/mmcblk0p2 /mnt/rpi-rootfs

Before we proceed, it is recommended to backup the current kernel image in case of any trouble.

sudo cp /mnt/rpi-boot/kernel7.img ~/

Then, to test our generated Linux image, we need to copy it to the boot partition and name the image in such a way that it gets selected for the next boot. From the Linux root directory where the kernel was compiled you might type:

sudo cp arch/arm/boot/zImage /mnt/rpi-boot/kernel7.img

Note: kernel7.img is the default kernel filename on Pi 2 and Pi 3. On Pi 1 and Pi Zero it is kernel.img.

Note 2: The Linux image name may be set on the config.txt file in the boot partition. To boot with a custom image name you may add kernel=<kernel file name> to it. For instance: echo kernel=my_kernel.img >> /mnt/rpi-boot/config.txt. See the Raspberry Pi boot options for more details.

The same idea applies to the device tree files. We may backup them:

mkdir ~/rpi-dtb-files
sudo cp /mnt/rpi-boot/*.dtb ~/rpi-dtb-files

Then copy the compiled dtb files in:

sudo cp arch/arm/boot/dts/*.dtb /mnt/rpi-boot/
sudo cp arch/arm/boot/dts/overlays/*.dtbo /mnt/rpi-boot/overlays/

To install the kernel modules, do:

sudo make INSTALL_MOD_PATH=/mnt/rpi-rootfs modules_install

Synchronize and umount the partitions to ensure data will persist on them:

rsync /mnt/rpi-boot
rsync /mnt/rpi-rootfs
sudo umount /mnt/rpi-boot
sudo umount /mnt/rpi-rootfs

For additional references on Raspberry Pi boot configurations, see the config-txt documentation.

We’re now ready to take the SD card out from the computer, attach it to the Raspberry Pi, and power the Pi on.

Test The Kernel Image

That’s the hour of truth. Attach the SD card to the Pi along with any peripherals you want (monitor, keyboard, etc.) so to be able to see what’s happening. Alternative ways of seeing what’s happening on the Raspberry Pi I’ve explored may be seen on this tutorial.

Power on and wait it to (hopefully) boot. If boot process completes normally, you may get a terminal to login. Once logged in, you may check the kernel version being used:

uname -r

That should have the same version number than the one at the Makefile in the base directory of the kernel you’ve compiled. Its should follow the format VERSION.PATCHLEVEL.SUBLEVEL

You may also see the installed modules at /lib/modules/<kernel version>. For instance, to see the installed IIO modules for kernel version 4.14.50-v7+ you may look for them:

ls /lib/modules/4.14.50-v7+/kernel/drivers/iio/

That’s it! You have successfully downloaded, configured, compiled, and tested a Linux kernel for Raspberry Pi. Congratulations!!!

—————————————————— —————————————————— ——————————————————

Revision History:

  • Rev1 (2019-09-03): Release
  • Rev2 (2019-09-19): Spelling corrections