This is a brief post relating some of my observations about the initialization of interrupt requests (IRQs) on the startup process of the Linux kernel. I intended to obtain a picture of the code execution flow that allocates an initializes interrupts for the ARM architecture, however, as I was creating a diagram of it, I tried not to dive so much into architecture-specific code.

My motivation to learn about IRQ functioning in the kernel was to help me implement a triggered buffer for a device driver, more specifically, an IIO triggered buffer. Roughly, an IIO triggered buffer is a buffer that stores data coming from a device when a certain condition (trigger) occurs. I admit that from IIO triggers to the IRQ subsystem there is a considerable focus shift, nevertheless, I’ve already visited many parts of the IRQ subsystem so let’s leave some notes with the hope they might at least be helpful in the future.

A diagram

The following diagram tries to give a big picture of the initialization of interrupts at Linux startup.

In the diagram:

  • Boxes are kernel functions.
  • Code execution order is from top to bottom.
  • Filled arrows pointing to boxes mean a function call.
  • Unfilled arrows point to comments about the above function or code flow.

Complementary notes for IRQ diagram

These are the same comments from the above diagram with some additional points. I place them here since putting them all on the diagram would make it more ugly and difficult to read.Also organized by function, comments are listed in the format:

<file where the function is implemented> - <function name>

<comments>

So here they are:

init/main.c - start_kernel()

Called during kernel boot/startup, this may be considered to be Linux’s main function (“C” code entry point). start_kernel() is non-architecture specific even though much of the work done during startup still has to be delegated to platform-specific functions.

start_kernel() is called from assembly code. You can see most (if not all) the calls to start_kernel() in the head.S file inside each architecture directory at arch//kernel/. Try executing the following commands from the kernel root dir.

grep -lr arch/ -e start_kernel | grep -E "head.S$"
find arch/ -name "*head.S" | xargs grep -n start_kernel -C 3 --color

For some references, see:

arch dependent - setup_arch()

Set machine_desc struct (available only during boot time). The machine_desc struct holds platform-specific information such as the architecture number, device tree (DT) compatible string, number of interrupt lines, etc. It also stores pointers to some initialization functions called to implement custom initialization at early kernel boot. Only a few architectures use machine_desc, mainly arm, powerpc, and arc.

arch/arm/kernel/devtree.c - setup_machine_fdt()

Return the machine_desc which best matches the device tree blob (DTB) compatible string.

drivers/of/fdt.c - of_flat_dt_match_machine()

Calls arch_get_next_match() to traverse the machine_desc table in order to find the description whose dt_compat best matches the compatible string at the DT root node.

Other work is done from the completition of setup_arch() to the next task I recognized as IRQ related. Hook for symmetric multiprocessing (SMP) CPUs are executed (smp_prepare_boot_cpu()), a minimum virtual file system is (vfs_caches_init_early()), some memory (mm_init()), and the scheduler are initialized (sched_init()), just to name a few. However, as we are focusing on IRQs, we jump to early_irq_init().

kernel/irq/irqdesc.c - early_irq_init()

Allocates and inserts IRQ descriptions (irq_desc) for as many preallocated interrupts the architecture defines the system should have.

arch dependent - arch_probe_nr_irqs()

Allows architecture code to probe hardware for the actual number of interrupt lines, overriding nr_irqs. Only called when CONFIG_SPARSE_IRQ is defined.

arch dependent - arch_early_irq_init()

Further early architecture-specific IRQ initialization tasks. Important for ia64, s390, and x86 archs.

arch dependent - init_IRQ()

Calls architecture-specific implementation (usually at arch//kernel/irq.c) to initialize existing interrupt lines. To see each of them, grep at arch dir

grep -nrI arch/ -e "__init init_IRQ"

or use C tags.

Many architectures such as arm, openrisc, mips, and riscv, call irqchip_init() in their implementation of init_IRQ().

grep -rl arch/ -e irqchip_init | grep -E ".*irq.*.c"

drivers/irqchip/irqchip.c - irqchip_init()

This seem to be arch independent since many different ones call it.

drivers/of/irq.c - of_irq_init()

Traverses the DT to initialize “interrupt-controller” nodes that are compatible with chips listed in the __irqchip_of_table. The __irqchip_of_table contains an array of struct of_device_id, each one describing an interrupt chip present on the board.

arch dependent - machine_desc->init_irq()

The struct machine_desc has a pointer to a function init_irq() which can be used to provide custom initialization of IRQs according to board-specific settings. Though, I have found few uses of it.

include/linux/acpi.h - acpi_probe_device_table()

I did not go deep on this call. Thinking about the initialization process, it seems reasonable to assume that the kernel also asks ACPI (if the hardware supports it) to discover devices not listed in the device tree (or all the devices in case no DTB was supplied).

Final considerations

I’m not 100% sure all this initialization actually goes serially, i.e., in one single execution thread. However, relying on a kernel comment which says that, “Full topology setup happens at smp_init() time - but meanwhile we still have a functioning scheduler.”, I believe all the start_kernel() does indeed runs in a single thread since smp_init() is called from within the init thread which in turn is started only at the end of start_kernel(), in a subcall of arch_call_rest_init().

The Linux kernel is not a static code base, either hardware stays the same over time. Implementation can change and so, in the future, things can be a little different from what I’ve described.

Finally, I’m not a kernel specialist and also, as a human being, I make mistakes. There is a sound possibility that I have misunderstood some steps of the IRQ initialization process, yet, I appreciate criticism (constructive ones preferably).

Aditional material that helped me understand more about IRQs:

Revision History:

  • Rev1 (2020-04-21): Release