What causes permission to be denied for mounting rootfs, home, messeage queue, kernel file system, during boot?

p.h.

My Fedora 27 x64 fails to boot after hard reset. It shows:

Failed to mount POSIX Message Queue File System,
Failed to start Remount and Kernel File Systems,
Failed to mount Kernel Debug File System,
Failed to mount Huge Pages File System [3]

and lots of other failures comes after these. See https://photos.app.goo.gl/qBUxT40zA2MTLTwO2

In all these cases

Failed at step EXEC spawning /usr/bin/mount: Permission denied

is given as a reason. How can it be? Doesn't it recognize it's own filesystems?

I have 3 kernels:

vmlinuz-4.14.16-300.fc27.x86_64
vmlinuz-4.15.13-300.fc27.x86_64
vmlinuz-4.15.14-300.fc27.x86_64

no matter which one I try to boot the same happens.

So far I have:

  1. Checked filesystem integrity with fsck. All partitions are clean.
  2. Checked disk health reported by SMART and performed both - short and long tests. Disk is perfectly healthy.
  3. Rebuilt initramfs. Mounted boot, proc, sys, dev in /mnt, chroot and sudo dracut.

Followed suggestions and:

  1. Performed fsck -f on /dev/mapper/fedora-home, got:

    tree extents for i-node 524820 (on level 2) could be narrower. Fix?<y>Y

Allowed to fix this.

And the same for /dev/mapper/fedora-root, /dev/sda1 (boot partition) confirmed they are clean. One more error of the same kind was found for an extra partition for data files.

  1. rpm -V --all | grep -v " [cg] " returned as follows:

.M.......    /run/libgpod
..5....T.    /var/lib/selinux/targeted/active/commit_num
.......T.    /var/lib/selinux/targeted/active/file_contexts
.......T.    /var/lib/selinux/targeted/active/homedir_template
S.5....T.    /var/lib/selinux/targeted/active/policy.kern
.M.....T.    /var/lib/selinux/targeted/active/seusers
.M.....T.    /var/lib/selinux/targeted/active/users_extra
.M.......    /var/run/pluto
not exists   /var/run/abrt
.M.......    /var/log/audit
not exists   /usr/lib/systemd/system-preset/85-display-manager.preset
S.5....T.    /usr/share/icons/Crux/icon-theme.cache
S.5....T.    /usr/share/icons/Mist/icon-theme.cache

  1. rpm -V "$(rpm -q --whatprovides /usr/bin/mount)"
    .M....G..  g /var/log/lastlog

  2. fixfiles check /usr

    libsemanage.semanage_make_sandbox: Error removing old sandbox directory /var/lib/selinux/targeted/tmp. (Read only file system). 
    genhomedircon: Could not begin transaction: Read only file system

Among many lines similar to to the one below:

Would relabel /usr/src/handbrake/trunk/build/contrib/lib from unconfined_u:object_r:usr_t:s0 to unconfined_u:object_r:lib_t:s0
a few interesting ones are:
Would relabel /usr/sbin/mount.nilfs2 from unconfined_u:object_r:bin_t:s0 to unconfined_u:object_r:mount_exec_t:s0
Would relabel /usr/sbin/umount.nilfs2 from unconfined_u:object_r:bin_t:s0 to unconfined_u:object_r:mount_exec_t:s0
Would relabel /usr/sbin/mkfs.nilfs2 from unconfined_u:object_r:bin_t:s0 to unconfined_u:object_r:fsadm_exec_t:s0

  1. Proved RAM works fine - memtest86 didn't find any errors during 3.5 passes and over 8h test time.

    9. Disabled SELinux (SELinux=disabled in /etc/selinux/config) and restarted. System started without any error! This proves problem is in SELinux policies. I believe I should start with checking those 6 top SELinux policies that have been changed somehow (see p. 5). The question is how to do it wisely.

  2. Checked local modifications to SELinux config files and file_contexts:

    semanage module -C -l
    Module name              Priority  Language
    semanage fcontext -C -l
    fcontext SELinux                                 type               Context
    /usr/bin/mount                                     all files          system_u:object_r:samba_share_t:s0
    /usr/share/dnfdaemon/dnfdaemon-system              all files          system_u:object_r:rpm_exec_t:s0
    /var/run/media/przemek/extra(/.*)?                 all files          system_u:object_r:samba_share_t:s0
    /var/www/html/photo                                all files          system_u:object_r:httpd_sys_rw_content_t:s0
    /var/www/html/photo/_cache                         all files          system_u:object_r:httpd_sys_rw_content_t:s0
    /var/www/html/photo/config                         all files          system_u:object_r:httpd_sys_rw_content_t:s0
    /var/www/html/photo/content                        all files          system_u:object_r:httpd_sys_rw_content_t:s0
    /var/www/html/photo/content/folders.json           all files          system_u:object_r:httpd_sys_rw_content_t:s0
    /var/www/html/photo/iv-config/language             all files          system_u:object_r:httpd_sys_rw_content_t:s0

Interestingly fcontext of the /usr/bin/mount has changed.

The system runs 24h/day as a simple home server (www, mail, etc.). From time to time (say once a few weeks) it freezes completely. HDD keeps writing something (repetitive, although irregular sound). No reaction to keyboard, mouse, remote SSH access. Many times I have tried to leave it overnight, but it does not recover, so I am forced to hard reset it each time this happens. This time I haven't waited, but hard reset it after just a few minutes. Unfortunately since then it cannot boot.

I remembered that a minute or less before the system froze Firefox message box appeared telling me that some script became irresponsive. I don't remember my choice (kill it/wait).

Hardware: Gigabyte GB-BACE-3160 Brix PC with Hitachi HTS725032A9A364 2.5" HDD and 4GB LPDDR3 RAM (default clock).

More details [here]

Failed to mount messages during boot Permission denied to mount POSIX Message File System, Kernel Debug File System Permission denied to mount Huge Pages File System, Remount Root and Kernel File Systems

p.h.

The problem was caused by improper file context of the /usr/bin/mount file: samba_share_t.

The file context change wasn't caused by some error due to hard reset, but... by my imprudent decision to follow the first suggestion of SELinux Alert Browser. See the screenshot below. enter image description here

This first suggestion was to change /usr/bin/mount file context to samba_share_t to allow smbd to access getattr.

The solution was:

  1. to delete invalid file context, restore default and relabel the file:
    [root@atlas ~]# ls -Z /usr/bin/mount
    system_u:object_r:samba_share_t:s0 /usr/bin/mount
    [root@atlas ~]# semanage fcontext -d /usr/bin/mount
    [root@atlas ~]# restorecon -v /usr/bin/mount
    Relabeled /usr/bin/mount from system_u:object_r:samba_share_t:s0 to system_u:object_r:mount_exec_t:s0
    [root@atlas ~]# ls -Z /usr/bin/mount
    system_u:object_r:mount_exec_t:s0 /usr/bin/mount
    
  2. reboot system.

It could be done in emergency console, but I have used the console to put SELinux into permissive mode, boot system and then change the file context as described above.

When I have checked modified SELinux contexts of files (see p. 10 of my initial post) I have noticed that context of the mount looked suspicious. At the moment I realized that shortly before the problem started I have imprudently followed the first suggestion of SELinux Alert Browser to change mount file context. The same suggestion appeared now, after system repair and restart, so I was able to attach the screenshot below.

Credit for @sourcejedi for pointing SELinux may be causing the problem and for his kind help!

Collected from the Internet

Please contact [email protected] to delete if infringement.

edited at
0

Comments

0 comments
Login to comment

Related

From Dev

What causes permission to be denied for mounting rootfs, home, messeage queue, kernel file system, during boot?

From Dev

What is a sync during kernel boot?

From Dev

Linux cp Permission denied on ntfs file system

From Dev

What causes "urlopen error [Errno 13] Permission denied" errors?

From Dev

cp: cannot create regular file '/home/Documents': Permission denied

From Dev

Permission denied when mounting Docker volume in OSX

From Dev

Permission denied from /boot

From Dev

Permission denied when accessing a file with acls on xfs file system

From Dev

Mounting nginx conf as a docker volume causes system error boot2docker

From Dev

Mounting nginx conf as a docker volume causes system error boot2docker

From Dev

Permission denied in opening device file at service in boot time

From Dev

Mount is denied mounting GlusterFS point at boot

From Dev

Problem to copy a file from HDFS to local system [copyToLocal: Permission denied]

From Dev

What Causes a File System to become Broken?

From Dev

reading a file: permission denied

From Dev

Permission denied to a 777 file

From Dev

Permission Denied To Delete File

From Dev

Permission Denied To Delete File

From Dev

Permission denied to a 777 file

From Dev

Unzip File: Permission Denied

From Dev

File creating permission Denied

From Dev

leapseconds file: Permission denied

From Dev

Which file in /proc gets read by the kernel during the boot up process?

From Dev

Mounting Encrypted VeraCrypt Drive during Boot

From Dev

What is the permission group 'others' in Linux file system?

From Dev

Permission Denied with mounting NFS with Webmin (Mac OSX + Ubuntu Setup)

From Dev

mounting network attached hdd( mybook) on ubuntu permission denied

From Dev

Security rules causes always getting "Permission denied"

From Dev

dmesg: read kernel buffer failed: Permission denied

Related Related

  1. 1

    What causes permission to be denied for mounting rootfs, home, messeage queue, kernel file system, during boot?

  2. 2

    What is a sync during kernel boot?

  3. 3

    Linux cp Permission denied on ntfs file system

  4. 4

    What causes "urlopen error [Errno 13] Permission denied" errors?

  5. 5

    cp: cannot create regular file '/home/Documents': Permission denied

  6. 6

    Permission denied when mounting Docker volume in OSX

  7. 7

    Permission denied from /boot

  8. 8

    Permission denied when accessing a file with acls on xfs file system

  9. 9

    Mounting nginx conf as a docker volume causes system error boot2docker

  10. 10

    Mounting nginx conf as a docker volume causes system error boot2docker

  11. 11

    Permission denied in opening device file at service in boot time

  12. 12

    Mount is denied mounting GlusterFS point at boot

  13. 13

    Problem to copy a file from HDFS to local system [copyToLocal: Permission denied]

  14. 14

    What Causes a File System to become Broken?

  15. 15

    reading a file: permission denied

  16. 16

    Permission denied to a 777 file

  17. 17

    Permission Denied To Delete File

  18. 18

    Permission Denied To Delete File

  19. 19

    Permission denied to a 777 file

  20. 20

    Unzip File: Permission Denied

  21. 21

    File creating permission Denied

  22. 22

    leapseconds file: Permission denied

  23. 23

    Which file in /proc gets read by the kernel during the boot up process?

  24. 24

    Mounting Encrypted VeraCrypt Drive during Boot

  25. 25

    What is the permission group 'others' in Linux file system?

  26. 26

    Permission Denied with mounting NFS with Webmin (Mac OSX + Ubuntu Setup)

  27. 27

    mounting network attached hdd( mybook) on ubuntu permission denied

  28. 28

    Security rules causes always getting "Permission denied"

  29. 29

    dmesg: read kernel buffer failed: Permission denied

HotTag

Archive