Skip to content

Instantly share code, notes, and snippets.

@Iksas
Created August 8, 2024 20:11
Show Gist options
  • Save Iksas/8cdf9bf1b0eff5d9322c6e1ec86ca15b to your computer and use it in GitHub Desktop.
Save Iksas/8cdf9bf1b0eff5d9322c6e1ec86ca15b to your computer and use it in GitHub Desktop.
systemd sandboxing

Fixing systemd sandboxing errors

Crash with status=31/SYS

user@ansible-test:~ $ journalctl -u mongod.service | tail
[...]
Aug 08 17:29:42 ansible-test systemd[1]: Started mongod.service - MongoDB Database Server.
Aug 08 17:29:43 ansible-test systemd[1]: mongod.service: Main process exited, code=killed, status=31/SYS
Aug 08 17:29:43 ansible-test systemd[1]: mongod.service: Failed with result 'signal'.

Error 31/SYS is mostly caused by incorrect SystemCallFilter settings. To see which system call caused the crash, perform the following steps:

Install the systemd-coredump package:

sudo apt install systemd-coredump

Systemd will now automatically store core dumps after a service crashes. Restart the service to cause one more crash to store a core dump:

sudo systemctl restart mongod.service

The journal should now indicate that a core dump has been saved (it says dumped instead of killed):

user@ansible-test:~ $ journalctl -u mongod.service | tail
[...]
Aug 08 17:44:25 ansible-test systemd[1]: Started mongod.service - MongoDB Database Server.
Aug 08 17:44:26 ansible-test systemd[1]: mongod.service: Main process exited, code=dumped, status=31/SYS
Aug 08 17:44:26 ansible-test systemd[1]: mongod.service: Failed with result 'core-dump'.

To load the most recent core dump into gdb, use the following command:

sudo coredumpctl debug

Once gdb has opened, use the info registers command to view the contents of the registers at the time of the crash:

(gdb) info registers
[...]
x7             0x7fff4b807b88      140734460099464
x8             0xe8                232
x9             0x0                 0
[...]

On arm64 systems, the system call number will be in register x8 (system call 232 in this case). On x64 systems, the system call number will be in register rax (I think).

Type quit to exit gdb, and look up the name of the system call: (arm64 / x64). In this example, system call number 232 belongs to the mincore system call.

Add a line to the [Service] section of the unit file to allow the required system call:

[Service]
SystemCallFilter=mincore

Reload the service files and restart the service to check if the error could be fixed:

sudo systemctl daemon-reload
sudo systemctl restart mongod.service
sudo systemctl status mongod.service
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment