What Do You Know about Your Linux System?

A look at the process to get insight into supported system calls and features and to assess how secure a system is and its runtime activity.

Apr 3rd, 2023 11:15am by Shuah Khan

Featued image for: What Do You Know about Your Linux System?

Do you know that Linux kernel-supported system calls and features are architecture dependent? Do you know that Linux kernel supports several hardening configuration options to secure your system? Let’s take a look at the process to get insight into supported system calls and features and to assess how secure a system is and its runtime activity.

System State Visualization

The kernel system state can be viewed as a combination of static and dynamic features and modules. Let’s first define what static and runtime system states are, and then explore how we can visualize the static and runtime system parts of the kernel. Static System View comprises system calls, features, static and dynamic modules enabled in the kernel configuration. Runtime System View comprises system calls, `ioctls` invoked and subsystems used during the runtime. A workload could load and unload modules and change the runtime system configuration to suit its needs by tuning system parameters. A few key points to remember:

Supported system calls and Linux kernel features are architecture-dependent. System call numbering is different on different architectures.
`auditd`, `checksyscalls.sh`, and `get_feat.pl` tools can be used to discover supported system calls and features.
Understanding Linux kernel-hardening configuration options and making sure they are enabled will make a system more secure.
Employing runtime tracing can shed light on the runtime system state.
Workloads could change the system state by loading and unloading dynamic modules and tuning system parameters.

How Do We Check Supported System Calls?

We have tools at our disposal to check for supported system calls and features. We can get the supported system call information using `auditd` utilities. `ausyscall –dump` from the `auditd` family of tools prints out the supported system calls on a system and allows mapping syscall names and numbers. You can install the `auditd` package on Debian-based systems:

sudo apt-get install auditd

Linux kernel tool `scripts/checksyscalls.sh` can be used to check if current architecture is missing any system calls compared to i386. Linux kernel tool `scripts/get_feat.pl` can be used to list the Kernel feature support matrix for an architecture.

Finding Supported System Calls

As mentioned earlier, `ausyscall` prints out supported system calls on a system and allows mapping syscalls names and numbers.

ausyscall --dump # prints out all supported system calls

`ausyscall` allows filtering on specific system calls or key strings. Let’s see what it shows when we invoke `ausyscall` with “open” and “time” options:

ausyscall open
open			2
mq_open		240
openat			257
    		perf_event_open	298
    		open_by_handle_at	304
    		open_tree		428
    		fsopen		430
    		pidfd_open		434
    		openat2		437

ausyscall time
getitimer		36
    		setitimer		38
    		gettimeofday		96
   		times			100
    		rt_sigtimedwait	128
    		utime			132
    		adjtimex		159
    		settimeofday		164
    		time			201
    		semtimedop		220
    		timer_create		222
    		timer_settime		223
    		timer_gettime		224
    		timer_getoverrun	225
    		timer_delete		226
    		clock_settime		227
    	clock_gettime		228
    		utimes			235
    		mq_timedsend	242
    		mq_timedreceive	243
    		futimesat		261
   		utimensat		280
    		timerfd_create	283
    		timerfd_settime	286
    		timerfd_gettime	287
    		clock_adjtime		305

Finding Unsupported System Calls

Understanding which system calls are not supported is important as well. As mentioned earlier, `scripts/checksyscalls.sh` checks missing system calls on current architecture compared to i386.

	checksyscalls.sh gcc
             warning: #warning syscall mmap2 not implemented [-Wcpp]
             warning: #warning syscall truncate64 not implemented [-Wcpp]
             warning: #warning syscall ftruncate64 not implemented [-Wcpp]
             warning: #warning syscall fcntl64 not implemented [-Wcpp]
             warning: #warning syscall sendfile64 not implemented [-Wcpp]
             warning: #warning syscall statfs64 not implemented [-Wcpp]
             warning: #warning syscall statfs64 not implemented [-Wcpp]
             warning: #warning syscall fadvise64_64 not implemented [-Wcpp]

Let’s check this against `ausyscall` now.

ausyscall map
      mmap	         9
      munmap		 11
      mremap		 25
      remap_file_pages	216

ausyscall trunc
      truncate		 76
      ftruncate		 77

As you can see, `ausyscall` shows mmap2, ftruncate64 and ftruncate64 aren’t implemented on this system. This matches what `checksyscalls.sh` shows.

Finding Supported Features

Let’s see how we can find the supported features on a system. `scripts/get_feat.pl` can be used to list the kernel feature support matrix for an architecture.

get_feat.pl list
get_feat.pl list –arch=arm64 lists

This script parses Documentation/features to find the support status information. It can be used to validate the contents of the files under Documentation/features or simply list them:

--arch option outputs features for an specific architecture, optionally filtering 
           for a single specific feature.
  --feat or --feature option outputs features for a single specific feature.

Here is how you can find if stack protector and thread-info-in-task features are supported:

scripts/get_feat.pl --arch=arm64 --feat=stackprotector list
        #
        # Kernel feature support matrix of the 'arm64' architecture:
        #
        debug/ stack protector       :  ok  |            HAVE_STACKPROTECTOR #
        arch supports compiler driven stack overflow protection

scripts/get_feat.pl --feat=thread-info-in-task list
        #
        # Kernel feature support matrix of the 'x86' architecture:
        #
        core/ thread-info-in-task  :  ok  |           THREAD_INFO_IN_TASK #
        arch makes use of the core kernel facility to embed thread_info in
        task_struct

Finding Kernel Module Status

`lsmod` command shows the kernel modules that are currently loaded. This program displays the contents of `/proc/modules`. Let’s pick `uvcvideo` module, which is found on most laptops:

lsmod | grep uvc
uvcvideo              126976  0
videobuf2_vmalloc      20480  1 uvcvideo
uvc                    16384  1 uvcvideo
videobuf2_v4l2         36864  1 uvcvideo
videodev              315392  2 videobuf2_v4l2,uvcvideo
videobuf2_common       65536  4 
                                      Videobuf2_vmalloc,videobuf2_v4l2,uvcvideo,videobuf2_memops
mc                     77824  4 videodev,videobuf2_v4l2,uvcvideo,videobuf2_common

You can see that `lsmod` shows `uvcvideo` and the modules it depends on, and how many modules are using them. `videobuf2_common` is in use by four other modules. In other words, this is the reference count for this module, and `rmmod` will refuse to unload it as long as the reference count is > 0. You can get the same information from `/proc/modules`:

 grep uvc  /proc/modules
  uvcvideo 126976 0 - Live 0x0000000000000000
  videobuf2_vmalloc 20480 1 uvcvideo, Live 0x0000000000000000
  uvc 16384 1 uvcvideo, Live 0x0000000000000000
  videobuf2_v4l2 36864 1 uvcvideo, Live 0x0000000000000000
  videodev 315392 2 uvcvideo,videobuf2_v4l2, Live 0x0000000000000000
  videobuf2_common 65536 4 uvcvideo,videobuf2_vmalloc,videobuf2_memops,videobuf2_v4l2, Live 0x0000000000000000
  mc 77824 4 uvcvideo,videobuf2_v4l2,videodev,videobuf2_common, Live 0x0000000000000000

The information is similar with a few extra fields. The address is the base address for the module in kernel virtual memory space. When run as a normal user, the address is all zeros. The same command when run as root will be as follows:

sudo grep uvc  /proc/modules
  uvcvideo 126976 0 - Live 0xffffffffc1c8b000
  videobuf2_vmalloc 20480 1 uvcvideo, Live 0xffffffffc167f000
  uvc 16384 1 uvcvideo, Live 0xffffffffc0ab0000
  videobuf2_v4l2 36864 1 uvcvideo, Live 0xffffffffc0a28000
  videodev 315392 2 uvcvideo,videobuf2_v4l2, Live 0xffffffffc16e9000
  videobuf2_common 65536 4 uvcvideo,videobuf2_vmalloc,videobuf2_memops,videobuf2_v4l2, Live 0xffffffffc094d000
  mc 77824 4 uvcvideo,videobuf2_v4l2,videodev,videobuf2_common, Live 0xffffffffc15eb000

Let’s now take a look at what `modinfo` shows us:

  /sbin/modinfo uvcvideo
  filename:       /lib/modules/6.3.0-rc2/kernel/drivers/media/usb/uvc/uvcvideo.ko
  license:        GPL
  description:    USB Video Class driver
  depends:        videobuf2-v4l2,videodev,mc,uvc,videobuf2-common,videobuf2-vmalloc
  retpoline:      Y
  intree:         Y
  name:           uvcvideo
  vermagic:       6.3.0-rc2 SMP preempt mod_unload modversions
  sig_id:         PKCS#7
  signer:         Build time autogenerated kernel key

This tells us that this module is built in the kernel repository signed with a build time autogenerated key. Let’s do one last sanity check on the system to see if the following two command outputs match:

  ps ax | wc -l
  ls -d /proc/* | grep [0-9]|wc -l

If they don’t match, examine your system closely. Kernel rootkits install their own ps, find, etc., utilities to mask their activity. The outputs match on my system. Do they match on yours?

Is My System as Secure as It Could Be?

The Linux kernel supports several hardening options to make the system secure. Let’s talk about `kconfig-hardened-check` tool sanity that can check kernel configuration for security. You can clone the latest `kconfig-hardened-check` repository:

git clone https://github.com/a13xp0p0v/kconfig-hardened-check.git
cd kconfig-hardened-check
bin/kconfig-hardened-check --config <config file> --cmdline /proc/cmdline

This will generate a detailed report of kernel security configuration and command line options that are enabled (OK) and the ones that aren’t (FAIL), and a summary line at the end:

[+] Config check is finished: 'OK' - 100 / 'FAIL' - 100

You will have to analyze the information to determine which options make sense to enable on your system.

Understanding System Runtime Activity

So far, we have looked for ways to find the static state of a system. Now let’s switch to the runtime state of a system. The Linux kernel event-tracing feature can help us with understanding the runtime state. Enabling event tracing gives insight into system runtime activity. This is a good way to identify which parts of the kernel are used at a higher level while the system is in and/or while a specific workload/process is running. Event tracing depends on the `CONFIG_EVENT_TRACING` option enabled. You can enable event tracing before starting workload/process. Event tracing allows you to enable and disable tracing on supported/available events at runtime. You can find available events, tracers and filter functions in the following files:

 /sys/kernel/debug/tracing/available_events
 /sys/kernel/debug/tracing/available_filter_functions
 /sys/kernel/debug/tracing/available_tracers

Now this is how you can enable tracing:

sudo echo 1 > /sys/kernel/debug/tracing/events/enable

Once the workload/process stops or when you decide you have the status you need, you can disable event tracing:

sudo echo 0 > /sys/kernel/debug/tracing/events/enable

You can find the tracing information in the file: /sys/kernel/debug/tracing Here is the information shown in this file:

 cat trace
 # tracer: nop
 #
 # entries-in-buffer/entries-written: 0/0   #P:16
 #
 #                                _-----=> irqs-off/BH-disabled
 #                               / _----=> need-resched
 #                              | / _---=> hardirq/softirq
 #                              || / _--=> preempt-depth
 #                              ||| / _-=> migrate-disable
 #                              |||| /     delay
 #           TASK-PID     CPU#  |||||  TIMESTAMP  FUNCTION
 #              | |         |   |||||     |         |

How Do We Use These Traces?

You can map the functions to system calls and other kernel features to get insight into the overall system activity while a workload/process is running.

Conclusion

As you can see, we have several tools and features at our disposal to get insight into system activity and assess it for security.

Shuah Khan is a Linux Foundation fellow and an upstream kernel maintainer and developer.