Last Updated: 2023-03-12 12:38:52 Sunday
-- TOC --
驱动，driver，在Linux内核内部，被称为mudole，内核模块！后者所表达的含义，应该更为准确！
即Linux Kernel源码解压后得到的树状目录结构，没啥神秘的，别被吓倒...
驱动程序跟Kernel版本密切相关，因为驱动程序要用到Kernel提供的一些全局符号（symbol），但Kernel在不同的release之间，并不能保证这些符号不发生变化。因此，编写驱动程序，要确定内核版本。最佳实践，自己download一个stable内核版本，编译安装并使用起来，然后在此相同源码树下开发驱动。尽量不使用发行版本的内核（除非你在为某个发行版本写驱动），因为据说这些内核都heavily patched，不是标准kernel环境，不是学习驱动编写的最佳环境。（编译安装Linux Kernel）
如果您的驱动程序需要支持很多不同版本的kernel，请#include <linux/version.h>，使用#ifdef来判断不同kernel版本。
即非in-tree module。
insmod out-of-tree module，会导致kernel tainted。
tainted kernel只是一种kernel的状态，在此状态下的bug可能得不到社区的支持。
A tainted kernel is one that is in an unsupported state because it cannot be guaranteed to function correctly. Most kernel developers will ignore bug reports involving tainted kernels, and community members may ask that you correct the tainting condition before they can proceed with diagnosing problems related to the kernel. In addition, some debugging functionality and API calls may be disabled when the kernel is tainted.
安装out-of-tree module会导致kernel进入tainted状态，但这个状态并不表示系统不能正常工作。（module使用kernel不支持的license，也会导致kernel进入tainted状态）
The tainted state is indicated by a series of flags which represent the various reasons a kernel cannot be trusted to work properly. The most common reason for the kernel to become tainted is loading a proprietary graphics driver from NVIDIA or AMD, in which case it is generally safe to ignore the condition. However, some scenarios that cause the kernel to become tainted may be indicative of more serious problems such as failing hardware. It is a good idea to examine system logs and the specific taint flags set to determine the underlying cause of the issue.
代码不开源的模块，本就没法debug。
This feature is intended to identify conditions which may make it difficult to properly troubleshoot a kernel problem. For example, a proprietary driver can cause problems that cannot be debugged reliably because its source code is not available and its effects cannot be determined. Likewise, if a serious kernel or hardware error had previously occurred, the integrity of the kernel space may have been compromised, meaning that any subsequent debug messages generated by the kernel may not be reliable.
只有重启系统，才能清除tainted flags。
Note that correcting the tainting condition alone does not remove the taint state because doing so does not change the fact that the kernel can no longer be relied on to work correctly or produce accurate debugging information. The system must be restarted to clear the taint flags.
检查当前Linux系统是否处于tainted状态
$ cat /proc/sys/kernel/tainted
0 # not be tainted
#include <linux/init.h>
#include <linux/module.h>
MODULE_LICENSE("GPL");
static int __init hello_init(void)
{
printk(KERN_ALERT "Hello, world\n");
return 0;
}
static void __exit hello_exit(void)
{
printk(KERN_ALERT "ByeBye, world\n");
}
module_init(hello_init);
module_exit(hello_exit);
内核空间的代码没有libc可以使用（没有runtime lib，它们是给用户态程序准备的），Linux Kernel提供了printk函数，与printf很相似，只是没有float number的支持。
MODULE_LICENSE("GPL");申明此模块的授权。
The specific licenses recognized by the kernel are
GPL(for any version of the GNU General Public License),GPL v2(for GPL version two only),GPL and additional rights,Dual BSD/GPL,Dual MPL/GPL, andProprietary. Unless your module is explicitly marked as being under a free license recognized by the kernel, it is assumed to be proprietary, and the kernel is tainted when the module is loaded. (more ininclude/linux/module.h)
必须准备一个Makefile文件才能编译内核模块，M要大写！不过内容可以很简单：
obj-m := hello_mod.o
如果module由多个文件组成，Makefile内容如下：
obj-m := module.o
module-objs := file1.o file2.o
然后在模块目录下输入命令：
$ make -C path/to/linux-5.14.14 M=$(pwd) modules
很快，我们就得到一个hello_mod.ko文件！
This command starts by changing its directory to the one provided with the
-Coption (that is, your kernel source tree directory). There it finds the kernel’s top-level makefile. TheM=option causes that makefile to move back into your module source directory before trying to build themodulestarget. This target, in turn, refers to the list of modules found in theobj-mvariable in your module's Makefile.
-C指向linux kernel tree。
M=指向待编译模块路径，没有-！
在一个指定的kernel内，在modules目标下，编译一个指定的module。
Module.symvers is missing$ make -C ../../linux-6.1.12/ M=$(pwd) modules
make: Entering directory '/home/xinlin/srcs/linux-6.1.12'
CC [M] /home/xinlin/srcs/kmtest/hello/hello_mod.o
MODPOST /home/xinlin/srcs/kmtest/hello/Module.symvers
WARNING: Module.symvers is missing.
Modules may not have dependencies or modversions.
You may get many unresolved symbol warnings.
WARNING: modpost: "__fentry__" [/home/xinlin/srcs/kmtest/hello/hello_mod.ko] undefined!
WARNING: modpost: "_printk" [/home/xinlin/srcs/kmtest/hello/hello_mod.ko] undefined!
WARNING: modpost: "__x86_return_thunk" [/home/xinlin/srcs/kmtest/hello/hello_mod.ko] undefined!
CC [M] /home/xinlin/srcs/kmtest/hello/hello_mod.mod.o
LD [M] /home/xinlin/srcs/kmtest/hello/hello_mod.ko
BTF [M] /home/xinlin/srcs/kmtest/hello/hello_mod.ko
make: Leaving directory '/home/xinlin/srcs/linux-6.1.12'
运行make [-j8] modules，就可以解决这个warning。
在用户态C语言项目中，函数默认是一个强符号，比如，在A模块中定义的函数，在B模块中声明之后（一般通过inlcude获得申明），就可直接调用。但Linux内核不支持这种方式，在A（内核）模块中定义的函数func，必须要使用EXPORT_SYMBOL导出，然后才能在B（内核）模块中正常调用。A导出的符号func保存在自己目录下的Module.symvers文件中，内核编译时会将这些符号收集汇总，最后生成源码树下的Module.symvers，供其他模块使用。同样的道理，如果一个内核模块在编译时，需要导出符号供其他人使用，也会生成该模块自己对应的Module.symvers文件，保存在该模块的当前目录下，这个文件的大小可以是0。
The Module.symvers is (re)generated when you (re)compile modules. Run make modules, and you should get a Module.symvers file at the root of the kernel tree. Note that if you only ran make and not make modules, you haven't built any modules yet. The symbols from the kernel itself (vmlinux or one of the architecture-dependent image formats) are in System.map.
You can simply make an empty one with
touch Modules.symvers. if your kernel doesn't use modversions (CRCs used by the operating system to help assure the modules match the kernel) and your module doesn't have any dependencies on other modules, this will be good enough to suppress the warning.
编译自己的模块时，编译系统会检查源码树下的Module.symvers文件，查看本模块使用的symbol，是否在此源码树下存在。只是看一看，不存在就给出warning，无法在此时链接。
$ modinfo hello_mod.ko
filename: /home/xinlin/srcs/kmtest/hello/hello_mod.ko
license: GPL
depends:
retpoline: Y
name: hello_mod
vermagic: 6.1.12 SMP preempt mod_unload
注意这个vermagic字符串，它有可能会是模块加载失败的原因！整个modinfo信息，除了filename这一行，其它信息都在hello_mod.ko的.modinfo section中。
install module
$ sudo insmod hello_mod.ko
insmod: ERROR: could not insert module hello_mod.ko: Invalid module format
相关命令：rmmod，lsmod
刚才hello_mod.ko加载失败了，我们可以用dmesg命令查看内核打印，了解失败的详细信息：
$ dmesg | tail -n5
...
[22076.099109] hello_mod: version magic '6.1.12 SMP preempt mod_unload ' should be '6.1.12-100.fc36.x86_64 SMP preempt mod_unload '
失败的原因：version magic不能与当前running系统匹配。我在标准6.1.12上编译内核模块，但却尝试在fedora上加载它。（这行打印，就是printk打出来的）
当前系统的version信息，用uname命令查看：
$ uname -r
6.1.12-100.fc36.x86_64
$ uname -a
Linux likecat 6.1.12-100.fc36.x86_64 #1 SMP PREEMPT_DYNAMIC Wed Feb 15 04:33:28 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
modprobe命令在安装模块的时候，会连同模块的依赖一切安装。
modprobe, like insmod, loads a module into the kernel. It differs in that it will look at the module to be loaded to see whether it references any symbols that are not currently defined in the kernel. If any such references are found, modprobe looks for other modules in the current module search path that define the relevant symbols. When modprobe finds those modules (which are needed by the module being loaded), it loads them into the kernel as well. If you use insmod in this situation instead, the command fails with an “unresolved symbols” message left in the system logfile.
modprobe会将module依赖module也一并load，而且它不是搜索当前路径，而是标准路径。（module之间也有层次依赖关系，module stacking）
/lib/modules/$(uname -r)目录下查找模块.ko后缀-f参数，强行加载模块，它是--force-vermagic和--force-modversion一起作用的效果。在fedora上测试modprobe的-f参数，没有效果，感觉是fedora给kernel打了补丁，禁用了此功能？
到这里，我们可怜的hello_mod.ko还不能成功加载，如下几个方式可以解决此问题：
本文选择切换源码树的方法，fedora系统。
$ sudo dnf install kernel-devel
这个命令将安装一套与当前系统匹配的，可以用来编译out-of-tree module的内核资源，安装位置为/usr/src/kernels/6.1.14-100.fc36.x86_64。（我的经历稍微曲折一点，安装kernel-devel 6.1.14时，running kernel是6.1.12，版本不匹配，dnf始终提示找不到6.1.12这个版本。此时，我注意到系统已经准备好了一次升级，能够将内核升级到6.1.14...）
然后，删除所有生成的文件，重新编译生成hello_mod.ko。此时的modinfo已经能够与系统对应上了。
$ make -C /lib/modules/6.1.14-100.fc36.x86_64/build/ M=$(pwd) modules
build这个路径，对应的就是上面的src路径，这似乎是个惯例！
$ sudo insmod hello_mod.ko
$ dmesg | tail -n1
[ 762.249977] Hello, world
$ sudo rmmod hello_mod
$ dmesg | tail -n1
[ 775.789854] ByeBye, world
似乎只有用ismod命令安装模块，必须要带上.ko后缀。
$ lsmod
Module Size Used by
hello_mod 16384 0
...
#include <linux/init.h>
#include <linux/module.h>
MODULE_LICENSE("GPL");
static char *whom = "world";
static int howmany = 1;
module_param(howmany, int, S_IRUGO);
module_param(whom, charp, S_IRUGO);
static int __init hello_init(void)
{
int i;
for (i=0; i<howmany; i++)
printk(KERN_ALERT "Hello, %s\n", whom);
return 0;
}
static void __exit hello_exit(void)
{
printk(KERN_ALERT "Goodbye, cruel world\n");
}
module_init(hello_init);
module_exit(hello_exit);
以上代码实现了模块参数功能，在insmod的时候，可以使用参数（modprobe通过配置文件/etc/modprobe.conf读取参数）。module_param定义在moduleparam.h文件中。
$ sudo insmod hello_mod.ko whom="xinlin" howmany=5
$ dmesg | tail -n5
[283751.863258] Hello, xinlin
[283751.863262] Hello, xinlin
[283751.863264] Hello, xinlin
[283751.863264] Hello, xinlin
[283751.863265] Hello, xinlin
S_IRUGO是个permission字段，见<linux/stat.h>文件。
If perm is set to 0 , there is no sysfs entry at all; otherwise, it appears under
/sys/modulewith the given set of permissions. Use S_IRUGO for a parameter that can be read by the world but cannot be changed; S_IRUGO|S_IWUSR allows root to change the parameter. Note that if a parameter is changed by sysfs, the value of that parameter as seen by your module changes, but your module is not notified in any other way. You should probably not make module parameters writable, unless you are prepared to detect the change and react accordingly.
$ pwd
/sys/module/hello_mod/parameters
$ ls
howmany whom
$ cat howmany
5
$ cat whom
xinlin
本文链接：https://cs.pynote.net/sf/linux/dd/202111275/
-- EOF --
-- MORE --