Working with Kernel Modules

• a device driver which adds support for new hardware; or,

• support for a file system such as btrfs or NFS.

Like the kernel itself, modules can take parameters that customize their behavior, though the default parameters work well in most cases. User-space tools can list the modules currently loaded into a running kernel; query all available modules for available parameters and module-specific information; and load or unload (remove) modules dynamically into or from a running kernel. Many of these utilities, which are provided by the kmod package, take module dependencies into account when performing operations so that manual dependency-tracking is rarely necessary.

On modern systems, kernel modules are automatically loaded by various mechanisms when the conditions call for it. However, there are occasions when it is necessary to load or unload modules manually, such as when one module is preferred over another although either could provide basic functionality, or when a module is misbehaving.

This chapter explains how to:

• use the user-space kmod utilities to display, query, load and unload kernel modules and their dependencies;

• set module parameters both dynamically on the command line and permanently so that you can customize the behavior of your kernel modules; and,

• load modules at boot time.

~]# dnf install kmod

Listing Currently-Loaded Modules

You can list all kernel modules that are currently loaded into the kernel by running the lsmod command, for example:

~]$ lsmod
Module                  Size  Used by
tcp_lp                 12663  0
bnep                   19704  2
bluetooth             372662  7 bnep
rfkill                 26536  3 bluetooth
fuse                   87661  3
ip6t_rpfilter          12546  1
ip6t_REJECT            12939  2
ipt_REJECT             12541  2
xt_conntrack           12760  7
ebtable_nat            12807  0
ebtable_broute         12731  0
bridge                110196  1 ebtable_broute
stp                    12976  1 bridge
llc                    14552  2 stp,bridge
ebtable_filter         12827  0
ebtables               30913  3 ebtable_broute,ebtable_nat,ebtable_filter
ip6table_nat           13015  1
nf_conntrack_ipv6      18738  5
nf_defrag_ipv6         34651  1 nf_conntrack_ipv6
nf_nat_ipv6            13279  1 ip6table_nat
ip6table_mangle        12700  1
ip6table_security      12710  1
ip6table_raw           12683  1
ip6table_filter        12815  1
ip6_tables             27025  5 ip6table_filter,ip6table_mangle,ip6table_security,ip6table_nat,ip6table_raw
iptable_nat            13011  1
nf_conntrack_ipv4      14862  4
nf_defrag_ipv4         12729  1 nf_conntrack_ipv4
nf_nat_ipv4            13263  1 iptable_nat
nf_nat                 21798  4 nf_nat_ipv4,nf_nat_ipv6,ip6table_nat,iptable_nat
[output truncated]

Each row of lsmod output specifies:

• the name of a kernel module currently loaded in memory;

• the amount of memory it uses; and,

• the sum total of processes that are using the module and other modules which depend on it, followed by a list of the names of those modules, if there are any. Using this list, you can first unload all the modules depending the module you want to unload. For more information, see Unloading a Module.

Finally, note that lsmod output is less verbose and considerably easier to read than the content of the /proc/modules pseudo-file.

Displaying Information About a Module

You can display detailed information about a kernel module by running the modinfo module_name command.

Example 1. Listing information about a kernel module with lsmod

To display information about the e1000e module, which is the Intel PRO/1000 network driver, run:

~]# modinfo e1000e
filename:       /lib/modules/3.17.4-302.fc21.x86_64/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko
version:        2.3.2-k
license:        GPL
description:    Intel(R) PRO/1000 Network Driver
author:         Intel Corporation, <linux.nics@intel.com>
srcversion:     2FBED3F5E2EF40112284D95
alias:          pci:v00008086d00001503sv*sd*bc*sc*i*
alias:          pci:v00008086d00001502sv*sd*bc*sc*i*
[some alias lines omitted]
alias:          pci:v00008086d0000105Esv*sd*bc*sc*i*
depends:        ptp
intree:         Y
vermagic:       3.17.4-302.fc21.x86_64 SMP mod_unload
signer:         Fedora kernel signing key
sig_key:        1F:C9:E6:8F:74:19:55:63:48:FD:EE:2F:DE:B7:FF:9D:A6:33:7B:BF
sig_hashalgo:   sha256
parm:           debug:Debug level (0=none,...,16=all) (int)
parm:           copybreak:Maximum size of packet that is copied to a new buffer on receive (uint)
parm:           TxIntDelay:Transmit Interrupt Delay (array of int)
parm:           TxAbsIntDelay:Transmit Absolute Interrupt Delay (array of int)
parm:           RxIntDelay:Receive Interrupt Delay (array of int)
parm:           RxAbsIntDelay:Receive Absolute Interrupt Delay (array of int)
parm:           InterruptThrottleRate:Interrupt Throttling Rate (array of int)
parm:           IntMode:Interrupt Mode (array of int)
parm:           SmartPowerDownEnable:Enable PHY smart power down (array of int)
parm:           KumeranLockLoss:Enable Kumeran lock loss workaround (array of int)
parm:           WriteProtectNVM:Write-protect NVM [WARNING: disabling this can lead to corrupted NVM] (array of int)
parm:           CrcStripping:Enable CRC Stripping, disable if your BMC needs the CRC (array of int)

Here are descriptions of a few of the fields in modinfo output:

filename

The absolute path to the .ko kernel object file. You can use modinfo -n as a shortcut command for printing only the filename field.

description

A short description of the module. You can use modinfo -d as a shortcut command for printing only the description field.

alias

The alias field appears as many times as there are aliases for a module, or is omitted entirely if there are none.

depends

This field contains a comma-separated list of all the modules this module depends on.

parm

Each parm field presents one module parameter in the form `parameter_name:description`, where:

• parameter_name is the exact syntax you should use when using it as a module parameter on the command line, or in an option line in a .conf file in the /etc/modprobe.d/ directory; and,

• description is a brief explanation of what the parameter does, along with an expectation for the type of value the parameter accepts (such as int, unit or array of int) in parentheses.

  Example 2. Listing module parameters

  You can list all parameters that the module supports by using the -p option. However, because useful value type information is omitted from modinfo -p output, it is more useful to run:

  ~]# modinfo e1000e | grep "^parm" | sort
  parm:           copybreak:Maximum size of packet that is copied to a new buffer on receive (uint)
  parm:           CrcStripping:Enable CRC Stripping, disable if your BMC needs the CRC (array of int)
  parm:           debug:Debug level (0=none,...,16=all) (int)
  parm:           InterruptThrottleRate:Interrupt Throttling Rate (array of int)
  parm:           IntMode:Interrupt Mode (array of int)
  parm:           KumeranLockLoss:Enable Kumeran lock loss workaround (array of int)
  parm:           RxAbsIntDelay:Receive Absolute Interrupt Delay (array of int)
  parm:           RxIntDelay:Receive Interrupt Delay (array of int)
  parm:           SmartPowerDownEnable:Enable PHY smart power down (array of int)
  parm:           TxAbsIntDelay:Transmit Absolute Interrupt Delay (array of int)
  parm:           TxIntDelay:Transmit Interrupt Delay (array of int)
  parm:           WriteProtectNVM:Write-protect NVM [WARNING: disabling this can lead to corrupted NVM] (array of int)

Loading a Module

To load a kernel module, run modprobe module_name as root. For example, to load the wacom module, run:

~]# modprobe wacom

By default, modprobe attempts to load the module from /lib/modules/*kernel_version*/kernel/drivers/. In this directory, each type of module has its own subdirectory, such as net/ and scsi/, for network and SCSI interface drivers respectively.

Some modules have dependencies, which are other kernel modules that must be loaded before the module in question can be loaded. The modprobe command always takes dependencies into account when performing operations. When you ask modprobe to load a specific kernel module, it first examines the dependencies of that module, if there are any, and loads them if they are not already loaded into the kernel. modprobe resolves dependencies recursively: it will load all dependencies of dependencies, and so on, if necessary, thus ensuring that all dependencies are always met.

You can use the -v (or --verbose) option to cause modprobe to display detailed information about what it is doing, which can include loading module dependencies.

Example 3. modprobe -v shows module dependencies as they are loaded

You can load the Fibre Channel over Ethernet module verbosely by typing the following at a shell prompt:

~]# modprobe -v fcoe
insmod /lib/modules/3.17.4-302.fc21.x86_64/kernel/drivers/scsi/scsi_transport_fc.ko.xz
insmod /lib/modules/3.17.4-302.fc21.x86_64/kernel/drivers/scsi/libfc/libfc.ko.xz
insmod /lib/modules/3.17.4-302.fc21.x86_64/kernel/drivers/scsi/fcoe/libfcoe.ko.xz
insmod /lib/modules/3.17.4-302.fc21.x86_64/kernel/drivers/scsi/fcoe/fcoe.ko.xz

In this example, you can see that modprobe loaded the scsi_tgt, scsi_transport_fc, libfc and libfcoe modules as dependencies before finally loading fcoe. Also note that modprobe used the more primitive insmod command to insert the modules into the running kernel.

Unloading a Module

You can unload a kernel module by running modprobe -r module_name as root. For example, assuming that the wacom module is already loaded into the kernel, you can unload it by running:

~]# modprobe -r wacom

However, this command will fail if a process is using:

• the wacom module;

• a module that wacom directly depends on, or;

• any module that wacom, through the dependency tree, depends on indirectly.

See Listing Currently-Loaded Modules for more information about using lsmod to obtain the names of the modules which are preventing you from unloading a certain module.

Example 4. Unloading a kernel module

For example, if you want to unload the firewire_ohci module, your terminal session might look similar to this:

~]# modinfo -F depends firewire_ohci
firewire-core
~]# modinfo -F depends firewire_core
crc-itu-t
~]# modinfo -F depends crc-itu-t

You have figured out the dependency tree (which does not branch in this example) for the loaded Firewire modules: firewire_ohci depends on firewire_core, which itself depends on crc-itu-t.

You can unload firewire_ohci using the modprobe -v -r module_name command, where -r is short for --remove and -v for --verbose:

~]# modprobe -r -v firewire_ohci
rmmod firewire_ohci
rmmod firewire_core
rmmod crc_itu_t

The output shows that modules are unloaded in the reverse order that they are loaded, given that no processes depend on any of the modules being unloaded.

Setting Module Parameters

Like the kernel itself, modules can also take parameters that change their behavior. Most of the time, the default ones work well, but occasionally it is necessary or desirable to set custom parameters for a module. Because parameters cannot be dynamically set for a module that is already loaded into a running kernel, there are two different methods for setting them.

1. You can unload all dependencies of the module you want to set parameters for, unload the module using modprobe -r, and then load it with modprobe along with a list of customized parameters. This method is often used when the module does not have many dependencies, or to test different combinations of parameters without making them persistent, and is the method covered in this section.

2. Alternatively, you can list the new parameters in an existing or newly created file in the /etc/modprobe.d/ directory. This method makes the module parameters persistent by ensuring that they are set each time the module is loaded, such as after every reboot or modprobe command. This method is covered in Persistent Module Loading, though the following information is a prerequisite.

Example 5. Supplying optional parameters when loading a kernel module

You can use modprobe to load a kernel module with custom parameters using the following command line format:

~]# modprobe module_name parameter=value

When loading a module with custom parameters on the command line, be aware of the following:

• You can enter multiple parameters and values by separating them with spaces.

• Some module parameters expect a list of comma-separated values as their argument. When entering the list of values, do not insert a space after each comma, or modprobe will incorrectly interpret the values following spaces as additional parameters.

• The modprobe command silently succeeds with an exit status of 0 if:

  • it successfully loads the module, or

  • the module is already loaded into the kernel.

    Thus, you must ensure that the module is not already loaded before attempting to load it with custom parameters. The modprobe command does not automatically reload the module, or alert you that it is already loaded.

Here are the recommended steps for setting custom parameters and then loading a kernel module. This procedure illustrates the steps using the e1000e module, which is the network driver for Intel PRO/1000 network adapters, as an example:

Loading a Kernel Module with Custom Parameters

1. First, ensure the module is not already loaded into the kernel:

   ~]# lsmod |grep e1000e
   ~]#

   Output would indicate that the module is already loaded into the kernel, in which case you must first unload it before proceeding. See Unloading a Module for instructions on safely unloading it.

2. Load the module and list all custom parameters after the module name. For example, if you wanted to load the Intel PRO/1000 network driver with the interrupt throttle rate set to 3000 interrupts per second for the first, second, and third instances of the driver, and turn on debug, you would run, as root:

   ~]# modprobe e1000e InterruptThrottleRate=3000,3000,3000 debug=1

   This example illustrates passing multiple values to a single parameter by separating them with commas and omitting any spaces between them.

Persistent Module Loading

As shown in Listing information about a kernel module with lsmod, many kernel modules are loaded automatically at boot time. You can specify additional modules to be loaded by the systemd-modules-load.service daemon by creating a *program*.conf file in the /etc/modules-load.d/ directory, where program is any descriptive name of your choice. The files in /etc/modules-load.d/ are text files that list the modules to be loaded, one per line.

Example 6. A Text File to Load a Module

To create a file to load the virtio-net.ko module, create a file /etc/modules-load.d/virtio-net.conf with the following content:

# Load virtio-net.ko at boot
virtio-net

See the modules-load.d(5) and systemd-modules-load.service(8) man pages for more information.

Signing Kernel Modules for Secure Boot

Fedora includes support for the UEFI Secure Boot feature, which means that Fedora can be installed and run on systems where UEFI Secure Boot is enabled. [1] When Secure Boot is enabled, the EFI operating system boot loaders, the Fedora kernel, and all kernel modules must be signed with a private key and authenticated with the corresponding public key. The Fedora distribution includes signed boot loaders, signed kernels, and signed kernel modules. In addition, the signed first-stage boot loader and the signed kernel include embedded Fedora public keys. These signed executable binaries and embedded keys enable Fedora to install, boot, and run with the Microsoft UEFI Secure Boot CA keys that are provided by the UEFI firmware on systems that support UEFI Secure Boot.[2]

The information provided in the following sections describes steps necessary to enable you to self-sign privately built kernel modules for use with Fedora on UEFI-based systems where Secure Boot is enabled. These sections also provide an overview of available options for getting your public key onto the target system where you want to deploy your kernel module.

Prerequisites

In order to enable signing of externally built modules, the tools listed in the following table are required to be installed on the system.

Kernel Module Authentication

In Fedora, when a kernel module is loaded, the module’s signature is checked using the public X.509 keys on the kernel’s system key ring, excluding those keys that are on the kernel’s system black list key ring.

Sources For Public Keys Used To Authenticate Kernel Modules

During boot, the kernel loads X.509 keys into the system key ring or the system black list key ring from a set of persistent key stores as shown in Sources For System Key Rings

Note that if the system is not UEFI-based or if UEFI Secure Boot is not enabled, then only the keys that are embedded in the kernel are loaded onto the system key ring and you have no ability to augment that set of keys without rebuilding the kernel. The system black list key ring is a list of X.509 keys which have been revoked. If your module is signed by a key on the black list then it will fail authentication even if your public key is in the system key ring.

To confirm if Secure Boot is enabled, enter a command as follows:

~]$ mokutil --sb-state
SecureBoot enabled

If Secure Boot is not enabled then the message Failed to read SecureBoot is displayed.

You can display information about the keys on the system key rings using the keyctl utility. The following is abbreviated example output from a Fedora system where UEFI Secure Boot is not enabled.

~]# keyctl list %:.builtin_trusted_keys
1 key in keyring:
265061799: ---lswrv   0   0 asymmetric: Fedora kernel signing key: ba8e2919f98f3f8e2e27541cde0d1f...

The following is abbreviated example output from a Fedora system where UEFI Secure Boot is enabled.

~]# keyctl list %:.builtin_trusted_keys
 5 keys in keyring:
 ...asymmetric: Microsoft Windows Production PCA 2011: a92902398e16c497...
 ...asymmetric: Fedora kernel signing key: ba8e2919f98f3f8e2e27541cde0d...
 ...asymmetric: Fedora Secure Boot CA: fde32599c2d61db1bf5807335d7b20e4...
 ...asymmetric: Red Hat Test Certifying CA: 08a0ef5800cb02fb587c12b4032...
 ...asymmetric: Microsoft Corporation UEFI CA 2011: 13adbf4309bd82709c8...

The above output shows the addition of two keys from the UEFI Secure Boot “db” keys plus the Fedora Secure Boot CA which is embedded in the shim.efi boot loader.

Kernel Module Authentication Requirements

If UEFI Secure Boot is enabled or if the module.sig_enforce kernel parameter has been specified, then only signed kernel modules that are authenticated using a key on the system key ring can be successfully loaded.[3] If UEFI Secure Boot is disabled and if the module.sig_enforce kernel parameter has not been specified, then unsigned kernel modules and signed kernel modules without a public key can be successfully loaded. This is summarized in Kernel Module Authentication Requirements for Loading.

Subsequent sections will describe how to generate a public and private X.509 key pair, how to use the private key to sign a kernel module, and how to enroll the public key into a source for the system key ring.

Generating a Public and Private X.509 Key Pair

You need to generate a public and private X.509 key pair that will be used to sign a kernel module after it has been built. The corresponding public key will be used to authenticate the kernel module when it is loaded.

1. The openssl tool can be used to generate a key pair that satisfies the requirements for kernel module signing in Fedora. Some of the parameters for this key generation request are best specified with a configuration file; follow the example below to create your own configuration file.

   ~]# cat << EOF > configuration_file.config
   [ req ]
   default_bits = 4096
   distinguished_name = req_distinguished_name
   prompt = no
   string_mask = utf8only
   x509_extensions = myexts
   [ req_distinguished_name ]
   O = Organization
   CN = Organization signing key
   emailAddress = E-mail address
   [ myexts ]
   basicConstraints=critical,CA:FALSE
   keyUsage=digitalSignature
   subjectKeyIdentifier=hash
   authorityKeyIdentifier=keyid
   EOF

2. After you have created the configuration file, you can create an X.509 public and private key pair. The public key will be written to the *public_key*.der file and the private key will be written to the *private_key*.priv file.

   ~]# openssl req -x509 -new -nodes -utf8 -sha256 -days 36500 \
    -batch -config configuration_file.config -outform DER \
    -out public_key.der \
    -keyout private_key.priv

3. Enroll your public key on all systems where you want to authenticate and load your kernel module.

Enrolling Public Key on Target System

When Fedora boots on a UEFI-based system with Secure Boot enabled, all keys that are in the Secure Boot db key database, but not in the dbx database of revoked keys, are loaded onto the system keyring by the kernel. The system keyring is used to authenticate kernel modules.

Factory Firmware Image Including Public Key

To facilitate authentication of your kernel module on your systems, consider requesting your system vendor to incorporate your public key into the UEFI Secure Boot key database in their factory firmware image.

Executable Key Enrollment Image Adding Public Key

It is possible to add a key to an existing populated and active Secure Boot key database. This can be done by writing and providing an EFI executable enrollment image. Such an enrollment image contains a properly formed request to append a key to the Secure Boot key database. This request must include data that is properly signed by the private key that corresponds to a public key that is already in the system’s Secure Boot Key Exchange Key (KEK) database. Additionally, this EFI image must be signed by a private key that corresponds to a public key that is already in the key database.

It is also possible to write an enrollment image that runs under Fedora. However, the Fedora image must be properly signed by a private key that corresponds to a public key that is already in the KEK database.

The construction of either type of key enrollment images requires assistance from the platform vendor.

System Administrator Manually Adding Public Key to the MOK List

The Machine Owner Key (MOK) facility is a feature that is supported by Fedora and can be used to augment the UEFI Secure Boot key database. When Fedora boots on a UEFI-enabled system with Secure Boot enabled, the keys on the MOK list are also added to the system keyring in addition to the keys from the key database. The MOK list keys are also stored persistently and securely in the same fashion as the Secure Boot key database keys, but these are two separate facilities. The MOK facility is supported by shim.efi, MokManager.efi, grubx64.efi, and the Fedora mokutil utility.

The major capability provided by the MOK facility is the ability to add public keys to the MOK list without needing to have the key chain back to another key that is already in the KEK database. However, enrolling a MOK key requires manual interaction by a physically present user at the UEFI system console on each target system. Nevertheless, the MOK facility provides an excellent method for testing newly generated key pairs and testing kernel modules signed with them.

Follow these steps to add your public key to the MOK list:

1. Request addition of your public key to the MOK list using a Fedora userspace utility:

   ~]# mokutil --import my_signing_key_pub.der

   You will be asked to enter and confirm a password for this MOK enrollment request.

2. Reboot the machine.

3. The pending MOK key enrollment request will be noticed by shim.efi and it will launch MokManager.efi to allow you to complete the enrollment from the UEFI console. You will need to enter the password you previously associated with this request and confirm the enrollment. Your public key is added to the MOK list, which is persistent.

Once a key is on the MOK list, it will be automatically propagated to the system key ring on this and subsequent boots when UEFI Secure Boot is enabled.

Signing Kernel Module with the Private Key

There are no extra steps required to prepare your kernel module for signing. You build your kernel module normally. Assuming an appropriate Makefile and corresponding sources, follow these steps to build your module and sign it:

1. Build your my_module.ko module the standard way:

   ~]# make -C /usr/src/kernels/$(uname -r) M=$PWD modules

2. Sign your kernel module with your private key. This is done with a C application. Note that the application requires that you provide both the files that contain your private and the public key as well as the kernel module file that you want to sign.

   ~]# /usr/src/kernels/$(uname -r)/scripts/sign-file \
    sha256 \
    my_signing_key.priv \
    my_signing_key_pub.der \
    my_module.ko

Your kernel module is in ELF image format and this application computes and appends the signature directly to the ELF image in your my_module.ko file. The modinfo utility can be used to display information about the kernel module’s signature, if it is present. For information on using the utility, see Displaying Information About a Module.

Note that this appended signature is not contained in an ELF image section and is not a formal part of the ELF image. Therefore, tools such as readelf will not be able to display the signature on your kernel module.

Your kernel module is now ready for loading. Note that your signed kernel module is also loadable on systems where UEFI Secure Boot is disabled or on a non-UEFI system. That means you do not need to provide both a signed and unsigned version of your kernel module.

Loading Signed Kernel Module

Once your public key is enrolled and is in the system keyring, the normal kernel module loading mechanisms will work transparently. In the following example, you will use mokutil to add your public key to the MOK list and you will manually load your kernel module with modprobe.

1. Optionally, you can verify that your kernel module will not load before you have enrolled your public key. First, verify what keys have been added to the system key ring on the current boot by running the keyctl list %:.builtin_trusted_keys as root. Since your public key has not been enrolled yet, it should not be displayed in the output of the command.

2. Request enrollment of your public key.

   ~]# mokutil --import my_signing_key_pub.der

3. Reboot, and complete the enrollment at the UEFI console.

   ~]# reboot

4. After the system reboots, verify the keys on the system key ring again.

   ~]# keyctl list %:.builtin_trusted_keys

5. You should now be able to load your kernel module successfully.

   ~]# modprobe -v my_module
   insmod /lib/modules/3.17.4-302.fc21.x86_64/extra/my_module.ko
   ~]# lsmod | grep my_module
   my_module 12425 0

Additional Resources

For more information on kernel modules and their utilities, see the following resources.

Manual Page Documentation

• lsmod(8) — The manual page for the lsmod command.

• modinfo(8) — The manual page for the modinfo command.

• modprobe(8) — The manual page for the modprobe command.

• rmmod(8) — The manual page for the rmmod command.

• ethtool(8) — The manual page for the ethtool command.

• mii-tool(8) — The manual page for the mii-tool command.

Installable and External Documentation

• Linux Loadable Kernel Module HOWTO — The Linux Loadable Kernel Module HOWTO from the Linux Documentation Project contains further information on working with kernel modules.

1. Fedora does not require the use of Secure Boot on UEFI systems.

2. Not all UEFI-based systems include support for Secure Boot.

3. Provided that the public key is not on the system black list key ring.