hyfr!

How Linux OS boots?

January 10, 2019 • ☕️☕️ 9 min read

When we boot our system it must start and we don’t know if we even need an OS at this point. But somehow later it should load any variant of an OS from some storage(can be floppy disk, pen drive etc). This pre-OS environment is a pretty nascent stage and having even proper disk I/O would be a luxury. So we have a program that initiates what we know as an OS running on the system, This program is called BIOS.

BIOS

On PC, the initial boot code is generally called BIOS(Basic input output system). The builtin BIOS is already configured for devices that live on the motherboard like disks, network interface,serial ports etc.

  • BIOS we use today are little smarter as they allow us to enter a configuration mode at boot time by holding down one or two special keys. On windows its ESC(multiple times) + F10 ( Yes! its that blue screen with grey tab and dropdowns)

  • These configuration allows to select which devices to boot from (Can boot from a pen drive, floppy disk, CD-ROM etc), configure and test storage devices, and select boot options, set passwords and security settings for system devices, configure power management features etc.

  • Once we decide on device to boot from then it BIOS loads first 512 bytes of disk (This is called MBR- master boot record)

  • MBR contains the program which tells the computer from which disk partition to load secondary boot program called Bootloader

Bootloaders

A program that loads an OS when a computer is turned on. There are two mainly used bootloaders in Linux :

  • GRUB (Grand Unified Bootloader: GRUB loads the kernel from /boot/vmlinuz-xx.x and plays a splash screen from the file /boot/grub/splash.xpm.gz.

    • Most of us including me have used this as this is default boot loader on Fedora, Red Hat, SUSE, windows etc.
    • By default GRUB reads its configuration from /boot/grub/grub.conf or grub.cfg
    $ cd /boot
    $ ls
    grub vmlinuz-3.13.0-44-generic otherfiles
    $ cd grub
    $ ls
    default  grubenv
    fonts     grub.cfg    
    $

    As grub.cfg is a pretty big file

    $ head -n 5 grub.cfg
    #
    # DO NOT EDIT THIS FILE
    #
    # It is automatically generated by grub-mkconfig using templates
    # from /etc/grub.d and settings from /etc/default/grub
    $

    GRUB command line options

    reboot Reboot the system
    root Specify a root device(partition)
    find Find a file on all mountable partitions
    kernel Load kernel from root device
    boot Boot system from specified kernel image
    help interactive help for commands


    For more details about GRUB and its command line options refer here

  • LILO(Traditional Linux Boot Loader) : It picks up its config from /etc/lilo.conf.

    • Used by ubuntu.
    • Slow compared to GRUB, If you have error in lilo.conf then lilo won’t discover the problem until halfway through.

Both the bootloaders allows command line flags passing to kernel. Example of flags

Option What it does?
init = /sbin/init Tells kernel to user /sbin/init as its init program.
init = /bin/bash Debug flag which tells to only start bash shell
init = /dev/cupcakes Tells kernel to user /dev/cupcakes as root device


These bootloaders then start the kernel which is described in below sections.

Bootstrapping

  • Process during which Kernel is loaded into main memory and beings to execute a variety of tasks. Boot time is a vulnerable period as errors in config files, missing equipments, damaged filesystems etc can prevent a computer from starting up.

  • When a computer is turned on, it executes boot code stored in ROM.That code figures out how to load and start kernel code.

  • Kernel looks over the system hardware and if all goes fine starts the system’s init process which is always process number 1.

    • Init is responsible for doing lot of steps like filesystem mount, starting system daemons etc.
    • Init does this with help of series of shell scripts called rc scripts. The manner in which these script are executed may vary from OS to OS.
  • Order of operations while booting

    • Load and init kernel
    • Hardware device identification and configuration.
    • Creation of kernel threads.
    • Execution of system startup scripts(RC scripts).
    • Multiuser operation
  • Admin have control only over the RC scripts using which she/he can bootstrap configuration.

  • systemd is a software suite that provides basic building blocks for a Linux OS. All the above startup software in provided in systemd beforehand. The systemctl command is used to examine and control the state of “systemd” system

systemctl [OPTION] [SERVICE]

Step 1 : Kernel Init

Kernel code is located usually at /boot/vmlinuz or related names(do cd /boot). In my system directory structure was :

hyfr@solodynamo:/boot$ ls
vmlinuz-3.16.0-4-amd64
extlinux
  • So kernel init has 2 steps i.e First a small program on ROM loads kernel code then kernel loads and performs memory tests to find out available RAM. Also kernel consists of static data structures so it knows beforehand how much memory it will need so it sets aside a fixed amount of real memory for itself when it starts.This memory is reserved from user-level process(processes we run). If you notice kernel prints on the console a message reporting the total amt of physical memory and amt available for user processes(hint: If u do ssh to some Unix server..this is the first message we see usually) ex:

    * Documentation:  https://help.ubuntu.com/
    
    System information as of Tue Jan  8 00:48:50 IST 2019
    
    System load:  0.02               Processes:           223
    Usage of /:   64.0% of xx.xxGB   Users logged in:     1
    Memory usage: 49%                IP address for eth0: xxx.xxx.xx.xx
    Swap usage:   0%
    
    Graph this data and manage this system at:
    https://landscape.canonical.com/
    
    Get cloud support with Ubuntu Advantage Cloud Guest:
    http://www.ubuntu.com/business/services/cloud

Step 2 : Hardware Configuration

There is already a device configuration present in kernel code. But this is minimum hardware it requires for execution and in addition kernel by using buses for devices asks the appropriate drivers for information. The drivers which doesn’t responds gets disabled.

Step 3 : Kernel Threads

Kernel creates several spontaneous processes(non-forks), init is followed by other memory and kernel handler processes . Import thing to note is that these threads aren’t your normal programmable threads i.e.

  • Kernel itself manages these threads which never block (each thread has its own Process Control Block).
  • These threads favor thread heavy processes but have high management footprint(slow in context switching).
  • User level threads are managed by a user level thread lib like libpthreads.a and have single Process Control Block so If one thread blocks then a process blocks. These threads have much lower management footprint.

Below are all the processes in init call which runs on kernel threads(complex names!)

init Runs rc scripts
kjournald Commits ext3 journal updates to disk
kswapd Swaps processes when physical memory is low
kreclaimd Reclaims memory pages not been used recently
ksoftirqd Handles soft interrupts
khubd Configures USB devices


  • Little intro to above k prefixed processes:

    • kjournald: ext3, or third extended filesystem commonly used by the Linux kernel. Its journaled file system i.e. keeps track of changes not yet committed to the file system’s main part by recording the intentions of such changes in a data structure known as a journal. It’s known to use a lot of CPU under certain loads.

    • kswapd: When physical memory becomes scarce then Linux memory management subsystem attempts to free physical pages. This task falls to the kernel swap daemon (kswapd).

      • It uses two variables, freepageshigh and freepageslow to decide if it should free some pages. So long as the number of free pages in the system remains above freepageshigh, the kernel swap daemon does nothing.
      • For seeing swap memory used by a processes run cat /proc/swaps No running process in my case
      hyfr@solodynamo:/proc$ cat /proc/swaps
      Filename                Type        Size    Used    Priority
      • For seeing all devices(memory disks etc) on which paging and swapping will be done run cat /etc/fstab
      hyfr@ankitsingh:/proc$ cat /etc/fstab
      # /etc/fstab: static file system information.
      proc    /proc    proc    nodev,noexec,nosuid    0    0
      /dev/vda1    /    ext3    errors=remount-ro    0    1
      
      //xx.xxx.xxx.xx/data_shared /data1 cifs uid=0,credentials=/home/hyfr/.smbcredentials,file_mode=0777,dir_mode=0777,noperm 0 0
      • Even if there is still available RAM, the Linux Kernel will move memory pages which are hardly ever used into swap space so if you don’t have enough memory, swap will be used quite often and noticeably more during any memory requirement spikes.

      • kswapd goes home if there is no swap? No, There are processes which have a memory that is backed by a file..so that data can be swapped to that file.

    • kreclaimd: A running system will eventually use all available page frames for inode entries (Everything in Linux is a file and each file has an inode, and this inode contains all of the file’s metadata) etc. Old pages which can be freed and invalidated for new uses before physical memory(RAM) is exhausted.

      • Page replacement policy used by keclaimd is not strict LRU based algorithm. LRU consists of two lists active_list and inactive_list
      • active_list: Contains all the active set of processes.
      • inactive_list: Contains set of processes which are candidates for reclamation. There is a lot of more things that can be discussed which would be better to do in a separate post.
    • ksoftirqd: Interrupts is one of the commonly used technique for multitasking. There are two types of interrupts soft(software) interrupt and hard(hardware) interrupts. No of hardware interrupts are limited to the number of interrupt request lines(IRQ) to the processor(CPU) from hardware connectors. ksoftirqd runs where the CPU is under heavy load of a software interrupt. Hardware interrupts are given priority over software interrupts.

      • Usually comes situations when IRQs come very very fast one after the other and the operating system cannot finish servicing one before another one arrives.
      • Such interrupts are queued by the operating system for later processing by a special internal process named ksoftirqd. To view run ps -e | grep “ksoftirqd”
      foodie@ankitsingh:~$ ps -e
        PID TTY          TIME CMD
          1 ?        00:02:43 systemd
          2 ?        00:00:00 kthreadd
          3 ?        02:31:22 ksoftirqd/0 (here!)
          5 ?        00:00:00 kworker/0:0H
    • _khubd : When a USB device is attached to the system async messages called USB request block(URB) are passed by calling *usbsubmit_urb* method.

      • When initially usbhubinit is called it creates daemon kernel thread called khubd. If interested here is the source
      • The khubd thread is responsible for configuring devices.

Summary of above writeup : BIOS ---calls---> Bootloaders ----calls---> kernel ---calls---> init(lot of things happen)

Further, we will look into init and rc scripts. You can leave your views, suggestion or appreciation by clicking on Discuss On Twitter.

Thanks! 🙌

The fault in my articles, they don't close themselves.

Discuss on Twitter