CPA conflicts detected when marking kernel_set_to_readonly as __ro_after_init #4
Description
vidal72[m] on IRC pointed out that 296db046b8 ("mark kernel_set_to_readonly as __ro_after_init") was causing problems since v4.20, probably due to 4046460b86 and f61c5ba288.
The following CPA conflict issues appeared in kernel logs:
kernel: CPA conflict Text RO: 0xffffffff87000000 - 0xffffffff871fffff PFN 1a9000 req 00000000000000e3 prevent 0000000000000002
kernel: WARNING: CPU: 1 PID: 1 at arch/x86/mm/pageattr.c:808 __change_page_attr_set_clr+0xc0b/0xfb0
kernel: Modules linked in:
kernel: CPU: 1 PID: 1 Comm: swapper/0 Tainted: G T 4.20.0-rc1 #1
kernel: RIP: 0010:__change_page_attr_set_clr+0xc0b/0xfb0
kernel: Code: 52 fc ff ff 48 3b 54 24 20 0f 87 47 fc ff ff 48 83 4c 24 08 02 41 f6 c6 02 0f 85 5d 16 00 00 4c 85 74 24 08 0f 84 37 fc ff ff <0f> 0b 41 80 4f 38 02 48 c7 c7 e8 5e 69 88 e8 02 a3 74 00 e9 ec f4
kernel: RSP: 0018:ffff9b6580cbbd60 EFLAGS: 00010202
kernel: RAX: 00000000001a9f45 RBX: 00000000001a9000 RCX: 0000000080000000
kernel: RDX: 00000000001a9c00 RSI: 0000000000000086 RDI: 0000000000000001
kernel: RBP: 0000000000000200 R08: 0000000000000001 R09: 000000000000029f
kernel: R10: 0000000000000002 R11: 00000000000000e1 R12: ffff916baa00b1c0
kernel: R13: ffffffff87000000 R14: 00000000000000e3 R15: ffff9b6580cbbe88
kernel: FS: 0000000000000000(0000) GS:ffff916c59480000(0000) knlGS:0000000000000000
kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
kernel: CR2: 00006806f64b2984 CR3: 00000001aa008001 CR4: 00000000003606e0
kernel: DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
kernel: DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
kernel: Call Trace:
kernel: ? 0xffffffff87000000
kernel: ? hugetlb_vm_op_pagesize+0xf/0x40
kernel: ? cpumask_next+0x18/0x20
kernel: ? purge_fragmented_blocks_allcpus+0x3d/0x210
kernel: change_page_attr_set_clr+0x143/0x2c0
kernel: ? 0xffffffff87000000
kernel: ? 0xffffffff87000000
kernel: set_memory_ro+0x27/0x30
kernel: ? 0xffffffff87000000
kernel: mark_rodata_ro+0x5b/0xb8
kernel: ? rest_init+0xbf/0xbf
kernel: kernel_init+0x2c/0x101
kernel: ret_from_fork+0x35/0x40
kernel: ---[ end trace f7aa2a08286c218f ]---
kernel: CPA conflict Rodata RO: 0xffff916b80000000 - 0xffff916bbfffffff PFN 180000 req 80000000000000e3 prevent 0000000000000002
kernel: CPA: Cannot fixup static protections for PUD split
kernel: CPA conflict Text RO: 0xffffffff87200000 - 0xffffffff873fffff PFN 1a9200 req 00000000000000e3 prevent 0000000000000002
kernel: CPA conflict Text RO: 0xffffffff87400000 - 0xffffffff875fffff PFN 1a9400 req 00000000000000e3 prevent 0000000000000002
kernel: CPA conflict Text RO: 0xffffffff87600000 - 0xffffffff877fffff PFN 1a9600 req 00000000000000e3 prevent 0000000000000002
kernel: CPA conflict Text RO: 0xffffffff87800000 - 0xffffffff879fffff PFN 1a9800 req 00000000000001e3 prevent 0000000000000002
kernel: CPA conflict Text RO: 0xffffffff87a00000 - 0xffffffff87bfffff PFN 1a9a00 req 00000000000000e3 prevent 0000000000000002
kernel: CPA conflict Text RO: 0xffffffff87c00000 - 0xffffffff87dfffff PFN 1a9c00 req 00000000000000e3 prevent 0000000000000002
kernel: CPA conflict Rodata RO: 0xffffffff87c00000 - 0xffffffff87dfffff PFN 1a9c00 req 00000000000000e3 prevent 0000000000000002
kernel: CPA conflict Rodata RO: 0xffff916ba9c00000 - 0xffff916ba9dfffff PFN 1a9c00 req 80000000000000e3 prevent 0000000000000002
kernel: CPA conflict Text RO: 0xffffffff87e00000 - 0xffffffff87ffffff PFN 1a9e00 req 00000000000000e3 prevent 0000000000000002
kernel: CPA conflict Rodata RO: 0xffffffff87e00000 - 0xffffffff87ffffff PFN 1a9e00 req 00000000000000e3 prevent 0000000000000002
kernel: CPA conflict Rodata RO: 0xffff916ba9e00000 - 0xffff916ba9ffffff PFN 1a9e00 req 80000000000000e3 prevent 0000000000000002
From what I understand, when calling set_memory_ro(), we arrive down __should_split_large_page(), then calling static_protections(old_prot, [...], CPA_CONFLICT) which checks old_prot for conflicts (using check_conflict()) with prot restrictions returned by e.g. protect_rodata().
However due to the aforementioned linux-hardened patch we have kernel_set_to_readonly=1 already, thus protect_rodata() returns __PAGE_RW. So we are checking that the existing mapping is already RO while we are actually in the process of making it RO.
So this explains the CPA conflict warnings. Then static_protections() returns chk_prot which is old_prot stripped from __PAGE_RW. Thus it triggers the WARN_ON_ONCE right below, and should_split_large_page() finally returns 1, meaning the large page is to be split unconditionally.
Note that before returning, it sets cpa->force_static_prot = 1;. This will make all following calls to split_set_pte() call static_protections([...], CPA_PROTECT) to "remove the invalid protection". Here's the reason for all the following CPA protect warning logs. The good news is that if we are splitting a PMD then all resulting pages are forced to be RO. However when splitting a PUD, the mapping cannot be fixed trivially so a warning is printed once and pages remain RW. vidal72[m] was observing this warning in their logs.
A similar issue was mentioned here.
I think this patch should be dropped from linux-hardened for now because it is semantically wrong and I don't see how it can be fixed cleanly. I also wonder whether the same issue is going to arise in grsecurity, especially when building without PAX_KERNEXEC.