summaryrefslogtreecommitdiff
Commit message (Collapse)AuthorAge
* UPSTREAM: arm64: avoid TLB conflict with CONFIG_RANDOMIZE_BASEMark Rutland2016-09-22
| | | | | | | | | | | | | | | | | | | | | | | | | | | | When CONFIG_RANDOMIZE_BASE is selected, we modify the page tables to remap the kernel at a newly-chosen VA range. We do this with the MMU disabled, but do not invalidate TLBs prior to re-enabling the MMU with the new tables. Thus the old mappings entries may still live in TLBs, and we risk violating Break-Before-Make requirements, leading to TLB conflicts and/or other issues. We invalidate TLBs when we uninsall the idmap in early setup code, but prior to this we are subject to issues relating to the Break-Before-Make violation. Avoid these issues by invalidating the TLBs before the new mappings can be used by the hardware. Fixes: f80fb3a3d508 ("arm64: add support for kernel ASLR") Cc: <stable@vger.kernel.org> # 4.6+ Acked-by: Ard Biesheuvel <ard.biesheuvel@linaro.org> Acked-by: Will Deacon <will.deacon@arm.com> Signed-off-by: Mark Rutland <mark.rutland@arm.com> Signed-off-by: Catalin Marinas <catalin.marinas@arm.com> Bug: 30369029 Patchset: kaslr-arm64-4.4 (cherry picked from commit fd363bd417ddb6103564c69cfcbd92d9a7877431) Signed-off-by: Jeff Vander Stoep <jeffv@google.com> Change-Id: I6c23ce55cdd8b66587b6787b8f28df8535e39f24
* UPSTREAM: arm64: Only select ARM64_MODULE_PLTS if MODULES=yCatalin Marinas2016-09-22
| | | | | | | | | | | | | | | | | | | | | | | | | | Selecting CONFIG_RANDOMIZE_BASE=y and CONFIG_MODULES=n fails to build the module PLTs support: CC arch/arm64/kernel/module-plts.o /work/Linux/linux-2.6-aarch64/arch/arm64/kernel/module-plts.c: In function ‘module_emit_plt_entry’: /work/Linux/linux-2.6-aarch64/arch/arm64/kernel/module-plts.c:32:49: error: dereferencing pointer to incomplete type ‘struct module’ This patch selects ARM64_MODULE_PLTS conditionally only if MODULES is enabled. Fixes: f80fb3a3d508 ("arm64: add support for kernel ASLR") Cc: <stable@vger.kernel.org> # 4.6+ Reported-by: Jeff Vander Stoep <jeffv@google.com> Acked-by: Ard Biesheuvel <ard.biesheuvel@linaro.org> Acked-by: Mark Rutland <mark.rutland@arm.com> Signed-off-by: Catalin Marinas <catalin.marinas@arm.com> Bug: 30369029 Patchset: kaslr-arm64-4.4 (cherry picked from commit b9c220b589daaf140f5b8ebe502c98745b94e65c) Signed-off-by: Jeff Vander Stoep <jeffv@google.com> Change-Id: I446cb3aa78f1c64b5aa1e2e90fda13f7d46cac33
* UPSTREAM: arm64: kasan: Use actual memory node when populating the kernel ↵Catalin Marinas2016-09-22
| | | | | | | | | | | | | | | | | | | | | | image shadow With the 16KB or 64KB page configurations, the generic vmemmap_populate() implementation warns on potential offnode page_structs via vmemmap_verify() because the arm64 kasan_init() passes NUMA_NO_NODE instead of the actual node for the kernel image memory. Fixes: f9040773b7bb ("arm64: move kernel image to base of vmalloc area") Signed-off-by: Catalin Marinas <catalin.marinas@arm.com> Reported-by: James Morse <james.morse@arm.com> Acked-by: Ard Biesheuvel <ard.biesheuvel@linaro.org> Acked-by: Mark Rutland <mark.rutland@arm.com> Bug: 30369029 Patchset: kaslr-arm64-4.4 (cherry picked from commit 2f76969f2eef051bdd63d38b08d78e790440b0ad) Signed-off-by: Jeff Vander Stoep <jeffv@google.com> Change-Id: I8985e5b4628a9c7076767d4565f7633635813b5c
* UPSTREAM: arm64: lse: deal with clobbered IP registers after branch via PLTArd Biesheuvel2016-09-22
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The LSE atomics implementation uses runtime patching to patch in calls to out of line non-LSE atomics implementations on cores that lack hardware support for LSE. To avoid paying the overhead cost of a function call even if no call ends up being made, the bl instruction is kept invisible to the compiler, and the out of line implementations preserve all registers, not just the ones that they are required to preserve as per the AAPCS64. However, commit fd045f6cd98e ("arm64: add support for module PLTs") added support for routing branch instructions via veneers if the branch target offset exceeds the range of the ordinary relative branch instructions. Since this deals with jump and call instructions that are exposed to ELF relocations, the PLT code uses x16 to hold the address of the branch target when it performs an indirect branch-to-register, something which is explicitly allowed by the AAPCS64 (and ordinary compiler generated code does not expect register x16 or x17 to retain their values across a bl instruction). Since the lse runtime patched bl instructions don't adhere to the AAPCS64, they don't deal with this clobbering of registers x16 and x17. So add them to the clobber list of the asm() statements that perform the call instructions, and drop x16 and x17 from the list of registers that are callee saved in the out of line non-LSE implementations. In addition, since we have given these functions two scratch registers, they no longer need to stack/unstack temp registers. Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org> [will: factored clobber list into #define, updated Makefile comment] Signed-off-by: Will Deacon <will.deacon@arm.com> Signed-off-by: Catalin Marinas <catalin.marinas@arm.com> Bug: 30369029 Patchset: kaslr-arm64-4.4 (cherry picked from commit 5be8b70af1ca78cefb8b756d157532360a5fd663) Signed-off-by: Jeff Vander Stoep <jeffv@google.com> Change-Id: Ia44a54eba315a47a6b8aaa2259b444e0139162c0
* UPSTREAM: arm64: mm: check at build time that PAGE_OFFSET divides the VA ↵Ard Biesheuvel2016-09-22
| | | | | | | | | | | | | | | | | | | | space evenly Commit 8439e62a1561 ("arm64: mm: use bit ops rather than arithmetic in pa/va translations") changed the boundary check against PAGE_OFFSET from an arithmetic comparison to a bit test. This means we now silently assume that PAGE_OFFSET is a power of 2 that divides the kernel virtual address space into two equal halves. So make that assumption explicit. Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org> Signed-off-by: Catalin Marinas <catalin.marinas@arm.com> Bug: 30369029 Patchset: kaslr-arm64-4.4 (cherry picked from commit 6d2aa549de1fc998581d216de3853aa131aa4446) Signed-off-by: Jeff Vander Stoep <jeffv@google.com> Change-Id: I8c3bc8cdb7d7f7dea092fd1a208b04583a141054
* UPSTREAM: arm64: kasan: Fix zero shadow mapping overriding kernel image shadowCatalin Marinas2016-09-22
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | With the 16KB and 64KB page size configurations, SWAPPER_BLOCK_SIZE is PAGE_SIZE and ARM64_SWAPPER_USES_SECTION_MAPS is 0. Since kimg_shadow_end is not page aligned (_end shifted by KASAN_SHADOW_SCALE_SHIFT), the edges of previously mapped kernel image shadow via vmemmap_populate() may be overridden by subsequent calls to kasan_populate_zero_shadow(), leading to kernel panics like below: ------------------------------------------------------------------------------ Unable to handle kernel paging request at virtual address fffffc100135068c pgd = fffffc8009ac0000 [fffffc100135068c] *pgd=00000009ffee0003, *pud=00000009ffee0003, *pmd=00000009ffee0003, *pte=00e0000081a00793 Internal error: Oops: 9600004f [#1] PREEMPT SMP Modules linked in: CPU: 0 PID: 1 Comm: swapper/0 Not tainted 4.5.0-rc4+ #1984 Hardware name: Juno (DT) task: fffffe09001a0000 ti: fffffe0900200000 task.ti: fffffe0900200000 PC is at __memset+0x4c/0x200 LR is at kasan_unpoison_shadow+0x34/0x50 pc : [<fffffc800846f1cc>] lr : [<fffffc800821ff54>] pstate: 00000245 sp : fffffe0900203db0 x29: fffffe0900203db0 x28: 0000000000000000 x27: 0000000000000000 x26: 0000000000000000 x25: fffffc80099b69d0 x24: 0000000000000001 x23: 0000000000000000 x22: 0000000000002000 x21: dffffc8000000000 x20: 1fffff9001350a8c x19: 0000000000002000 x18: 0000000000000008 x17: 0000000000000147 x16: ffffffffffffffff x15: 79746972100e041d x14: ffffff0000000000 x13: ffff000000000000 x12: 0000000000000000 x11: 0101010101010101 x10: 1fffffc11c000000 x9 : 0000000000000000 x8 : fffffc100135068c x7 : 0000000000000000 x6 : 000000000000003f x5 : 0000000000000040 x4 : 0000000000000004 x3 : fffffc100134f651 x2 : 0000000000000400 x1 : 0000000000000000 x0 : fffffc100135068c Process swapper/0 (pid: 1, stack limit = 0xfffffe0900200020) Call trace: [<fffffc800846f1cc>] __memset+0x4c/0x200 [<fffffc8008220044>] __asan_register_globals+0x5c/0xb0 [<fffffc8008a09d34>] _GLOBAL__sub_I_65535_1_sunrpc_cache_lookup+0x1c/0x28 [<fffffc8008f20d28>] kernel_init_freeable+0x104/0x274 [<fffffc80089e1948>] kernel_init+0x10/0xf8 [<fffffc8008093a00>] ret_from_fork+0x10/0x50 ------------------------------------------------------------------------------ This patch aligns kimg_shadow_start and kimg_shadow_end to SWAPPER_BLOCK_SIZE in all configurations. Fixes: f9040773b7bb ("arm64: move kernel image to base of vmalloc area") Signed-off-by: Catalin Marinas <catalin.marinas@arm.com> Acked-by: Mark Rutland <mark.rutland@arm.com> Acked-by: Ard Biesheuvel <ard.biesheuvel@linaro.org> Bug: 30369029 Patchset: kaslr-arm64-4.4 (cherry picked from commit 2776e0e8ef683a42fe3e9a5facf576b73579700e) Signed-off-by: Jeff Vander Stoep <jeffv@google.com> Change-Id: I13a6b38aefbeddd20bc87cb1382f2787bbc5cf9c
* UPSTREAM: arm64: consistently use p?d_set_hugeMark Rutland2016-09-22
| | | | | | | | | | | | | | | | | | | | | | | | | | Commit 324420bf91f60582 ("arm64: add support for ioremap() block mappings") added new p?d_set_huge functions which do the hard work to generate and set a correct block entry. These differ from open-coded huge page creation in the early page table code by explicitly setting the P?D_TYPE_SECT bits (which are implicitly retained by mk_sect_prot() for any valid prot), but are otherwise identical (and cannot fail on arm64). For simplicity and consistency, make use of these in the initial page table creation code. Signed-off-by: Mark Rutland <mark.rutland@arm.com> Cc: Ard Biesheuvel <ard.biesheuvel@linaro.org> Cc: Will Deacon <will.deacon@arm.com> Signed-off-by: Catalin Marinas <catalin.marinas@arm.com> Bug: 30369029 Patchset: kaslr-arm64-4.4 (cherry picked from commit c661cb1c537e2364bfdabb298fb934fd77445e98) Signed-off-by: Jeff Vander Stoep <jeffv@google.com> Change-Id: I25e58a1626831c2c709abcded989d1770fea851c
* UPSTREAM: arm64: fix KASLR boot-time I-cache maintenanceMark Rutland2016-09-22
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Commit f80fb3a3d50843a4 ("arm64: add support for kernel ASLR") missed a DSB necessary to complete I-cache maintenance in the primary boot path, and hence stale instructions may still be present in the I-cache and may be executed until the I-cache maintenance naturally completes. Since commit 8ec41987436d566f ("arm64: mm: ensure patched kernel text is fetched from PoU"), all CPUs invalidate their I-caches after their MMU is enabled. Prior a CPU's MMU having been enabled, arbitrary lines may have been fetched from the PoC into I-caches. We never patch text expected to be executed with the MMU off. Thus, it is unnecessary to perform broadcast I-cache maintenance in the primary boot path. This patch reduces the scope of the I-cache maintenance to the local CPU, and adds the missing DSB with similar scope, matching prior maintenance in the primary boot path. Signed-off-by: Mark Rutland <mark.rutland@arm.com> Acked-by: Ard Biesehvuel <ard.biesheuvel@linaro.org> Cc: Will Deacon <will.deacon@arm.com> Signed-off-by: Catalin Marinas <catalin.marinas@arm.com> Bug: 30369029 Patchset: kaslr-arm64-4.4 (cherry picked from commit b90b4a608ea2401cc491828f7a385edd2e236e37) Signed-off-by: Jeff Vander Stoep <jeffv@google.com> Change-Id: Ic66b5fec29867b86782ad6c3243642afc1f40080
* UPSTREAM: arm64: hugetlb: partial revert of 66b3923a1a0fWill Deacon2016-09-22
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Commit 66b3923a1a0f ("arm64: hugetlb: add support for PTE contiguous bit") introduced support for huge pages using the contiguous bit in the PTE as opposed to block mappings, which may be slightly unwieldy (512M) in 64k page configurations. Unfortunately, this support has resulted in some late regressions when running the libhugetlbfs test suite with 64k pages and CONFIG_DEBUG_VM as a result of a BUG: | readback (2M: 64): ------------[ cut here ]------------ | kernel BUG at fs/hugetlbfs/inode.c:446! | Internal error: Oops - BUG: 0 [#1] SMP | Modules linked in: | CPU: 7 PID: 1448 Comm: readback Not tainted 4.5.0-rc7 #148 | Hardware name: linux,dummy-virt (DT) | task: fffffe0040964b00 ti: fffffe00c2668000 task.ti: fffffe00c2668000 | PC is at remove_inode_hugepages+0x44c/0x480 | LR is at remove_inode_hugepages+0x264/0x480 Rather than revert the entire patch, simply avoid advertising the contiguous huge page sizes for now while people are actively working on a fix. This patch can then be reverted once things have been sorted out. Cc: David Woods <dwoods@ezchip.com> Reported-by: Steve Capper <steve.capper@arm.com> Signed-off-by: Will Deacon <will.deacon@arm.com> Bug: 30369029 Patchset: kaslr-arm64-4.4 (cherry picked from commit ff7925848b50050732ac0401e0acf27e8b241d7b) Signed-off-by: Jeff Vander Stoep <jeffv@google.com> Change-Id: I3a9751fa79b2d2871dbdc06ea1aa3d1336bb4f4f
* UPSTREAM: arm64: make irq_stack_ptr more robustYang Shi2016-09-22
| | | | | | | | | | | | | | | | | | | Switching between stacks is only valid if we are tracing ourselves while on the irq_stack, so it is only valid when in current and non-preemptible context, otherwise is is just zeroed off. Fixes: 132cd887b5c5 ("arm64: Modify stack trace and dump for use with irq_stack") Acked-by: James Morse <james.morse@arm.com> Tested-by: James Morse <james.morse@arm.com> Signed-off-by: Yang Shi <yang.shi@linaro.org> Signed-off-by: Will Deacon <will.deacon@arm.com> Bug: 30369029 Patchset: kaslr-arm64-4.4 (cherry picked from commit a80a0eb70c358f8c7dda4bb62b2278dc6285217b) Signed-off-by: Jeff Vander Stoep <jeffv@google.com> Change-Id: I431d3d5e8e1f556ddfef283af88dd2f63b825f7c
* UPSTREAM: arm64: efi: invoke EFI_RNG_PROTOCOL to supply KASLR randomnessArd Biesheuvel2016-09-22
| | | | | | | | | | | | | | | | | | | | | | Since arm64 does not use a decompressor that supplies an execution environment where it is feasible to some extent to provide a source of randomness, the arm64 KASLR kernel depends on the bootloader to supply some random bits in the /chosen/kaslr-seed DT property upon kernel entry. On UEFI systems, we can use the EFI_RNG_PROTOCOL, if supplied, to obtain some random bits. At the same time, use it to randomize the offset of the kernel Image in physical memory. Reviewed-by: Matt Fleming <matt@codeblueprint.co.uk> Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org> Signed-off-by: Catalin Marinas <catalin.marinas@arm.com> Bug: 30369029 Patchset: kaslr-arm64-4.4 (cherry picked from commit 2b5fe07a78a09a32002642b8a823428ade611f16) Signed-off-by: Jeff Vander Stoep <jeffv@google.com> Change-Id: I9cb7ae5727dfdf3726b1c9544bce74722ec77bbd
* UPSTREAM: efi: stub: use high allocation for converted command lineArd Biesheuvel2016-09-22
| | | | | | | | | | | | | | | | | | | | | Before we can move the command line processing before the allocation of the kernel, which is required for detecting the 'nokaslr' option which controls that allocation, move the converted command line higher up in memory, to prevent it from interfering with the kernel itself. Since x86 needs the address to fit in 32 bits, use UINT_MAX as the upper bound there. Otherwise, use ULONG_MAX (i.e., no limit) Reviewed-by: Matt Fleming <matt@codeblueprint.co.uk> Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org> Signed-off-by: Catalin Marinas <catalin.marinas@arm.com> Bug: 30369029 Patchset: kaslr-arm64-4.4 (cherry picked from commit 48fcb2d0216103d15306caa4814e2381104df6d8) Signed-off-by: Jeff Vander Stoep <jeffv@google.com> Change-Id: Ie959355658d3f2f1819bee842c77cc5eef54b8e7
* UPSTREAM: efi: stub: add implementation of efi_random_alloc()Ard Biesheuvel2016-09-22
| | | | | | | | | | | | | | | | | | | | | | | | This implements efi_random_alloc(), which allocates a chunk of memory of a certain size at a certain alignment, and uses the random_seed argument it receives to randomize the address of the allocation. This is implemented by iterating over the UEFI memory map, counting the number of suitable slots (aligned offsets) within each region, and picking a random number between 0 and 'number of slots - 1' to select the slot, This should guarantee that each possible offset is chosen equally likely. Suggested-by: Kees Cook <keescook@chromium.org> Reviewed-by: Matt Fleming <matt@codeblueprint.co.uk> Reviewed-by: Kees Cook <keescook@chromium.org> Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org> Signed-off-by: Catalin Marinas <catalin.marinas@arm.com> Bug: 30369029 Patchset: kaslr-arm64-4.4 (cherry picked from commit 2ddbfc81eac84a299cb4747a8764bc43f23e9008) Signed-off-by: Jeff Vander Stoep <jeffv@google.com> Change-Id: I8f59e3e91a71c752d69fd08ca43a890977c82919
* BACKPORT: efi: stub: implement efi_get_random_bytes() based on EFI_RNG_PROTOCOLArd Biesheuvel2016-09-22
| | | | | | | | | | | | | | | | This exposes the firmware's implementation of EFI_RNG_PROTOCOL via a new function efi_get_random_bytes(). Reviewed-by: Matt Fleming <matt@codeblueprint.co.uk> Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org> Signed-off-by: Catalin Marinas <catalin.marinas@arm.com> Bug: 30369029 Patchset: kaslr-arm64-4.4 (cherry picked from commit e4fbf4767440472f9d23b0f25a2b905e1c63b6a8) Signed-off-by: Jeff Vander Stoep <jeffv@google.com> Change-Id: Id46036b78c2efd223b6cd5488e512fd93e8f597d
* BACKPORT: arm64: kaslr: randomize the linear regionArd Biesheuvel2016-09-22
| | | | | | | | | | | | | | | | | | | | When KASLR is enabled (CONFIG_RANDOMIZE_BASE=y), and entropy has been provided by the bootloader, randomize the placement of RAM inside the linear region if sufficient space is available. For instance, on a 4KB granule/3 levels kernel, the linear region is 256 GB in size, and we can choose any 1 GB aligned offset that is far enough from the top of the address space to fit the distance between the start of the lowest memblock and the top of the highest memblock. Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org> Signed-off-by: Catalin Marinas <catalin.marinas@arm.com> Bug: 30369029 Patchset: kaslr-arm64-4.4 (cherry picked from commit 031a4213c11a5db475f528c182f7b3858df11db) Signed-off-by: Jeff Vander Stoep <jeffv@google.com> Change-Id: I272de8ee358351d95eacc7dc5f47600adec3e813
* UPSTREAM: arm64: mm: treat memstart_addr as a signed quantityArd Biesheuvel2016-09-22
| | | | | | | | | | | | | | | | | | | | | | | Commit c031a4213c11 ("arm64: kaslr: randomize the linear region") implements randomization of the linear region, by subtracting a random multiple of PUD_SIZE from memstart_addr. This causes the virtual mapping of system RAM to move upwards in the linear region, and at the same time causes memstart_addr to assume a value which may be negative if the offset of system RAM in the physical space is smaller than its offset relative to PAGE_OFFSET in the virtual space. Since memstart_addr is effectively an offset now, redefine its type as s64 so that expressions involving shifting or division preserve its sign. Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org> Signed-off-by: Catalin Marinas <catalin.marinas@arm.com> Bug: 30369029 Patchset: kaslr-arm64-4.4 (cherry picked from commit 020d044f66874eba058ce8264fc550f3eca67879) Signed-off-by: Jeff Vander Stoep <jeffv@google.com> Change-Id: I0482ebc13baaa9005cf372795e656c2417be9d1c
* UPSTREAM: arm64: vmemmap: use virtual projection of linear regionArd Biesheuvel2016-09-22
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Commit dd006da21646 ("arm64: mm: increase VA range of identity map") made some changes to the memory mapping code to allow physical memory to reside at an offset that exceeds the size of the virtual mapping. However, since the size of the vmemmap area is proportional to the size of the VA area, but it is populated relative to the physical space, we may end up with the struct page array being mapped outside of the vmemmap region. For instance, on my Seattle A0 box, I can see the following output in the dmesg log. vmemmap : 0xffffffbdc0000000 - 0xffffffbfc0000000 ( 8 GB maximum) 0xffffffbfc0000000 - 0xffffffbfd0000000 ( 256 MB actual) We can fix this by deciding that the vmemmap region is not a projection of the physical space, but of the virtual space above PAGE_OFFSET, i.e., the linear region. This way, we are guaranteed that the vmemmap region is of sufficient size, and we can even reduce the size by half. Cc: <stable@vger.kernel.org> Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org> Signed-off-by: Will Deacon <will.deacon@arm.com> Bug: 30369029 Patchset: kaslr-arm64-4.4 (cherry picked from commit dfd55ad85e4a7fbaa82df12467515ac3c81e8a3e) Signed-off-by: Jeff Vander Stoep <jeffv@google.com> Change-Id: I8112d910f9659941dab6de5b3791f395150c77f1
* BACKPORT: arm64: add support for kernel ASLRArd Biesheuvel2016-09-22
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This adds support for KASLR is implemented, based on entropy provided by the bootloader in the /chosen/kaslr-seed DT property. Depending on the size of the address space (VA_BITS) and the page size, the entropy in the virtual displacement is up to 13 bits (16k/2 levels) and up to 25 bits (all 4 levels), with the sidenote that displacements that result in the kernel image straddling a 1GB/32MB/512MB alignment boundary (for 4KB/16KB/64KB granule kernels, respectively) are not allowed, and will be rounded up to an acceptable value. If CONFIG_RANDOMIZE_MODULE_REGION_FULL is enabled, the module region is randomized independently from the core kernel. This makes it less likely that the location of core kernel data structures can be determined by an adversary, but causes all function calls from modules into the core kernel to be resolved via entries in the module PLTs. If CONFIG_RANDOMIZE_MODULE_REGION_FULL is not enabled, the module region is randomized by choosing a page aligned 128 MB region inside the interval [_etext - 128 MB, _stext + 128 MB). This gives between 10 and 14 bits of entropy (depending on page size), independently of the kernel randomization, but still guarantees that modules are within the range of relative branch and jump instructions (with the caveat that, since the module region is shared with other uses of the vmalloc area, modules may need to be loaded further away if the module region is exhausted) Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org> Signed-off-by: Catalin Marinas <catalin.marinas@arm.com> Bug: 30369029 Patchset: kaslr-arm64-4.4 (cherry picked from commit f80fb3a3d50843a401dac4b566b3b131da8077a2) Signed-off-by: Jeff Vander Stoep <jeffv@google.com> Change-Id: I3f5fafa4e92e5ff39259d57065541366237eb021
* UPSTREAM: arm64: add support for building vmlinux as a relocatable PIE binaryArd Biesheuvel2016-09-22
| | | | | | | | | | | | | | | | | | This implements CONFIG_RELOCATABLE, which links the final vmlinux image with a dynamic relocation section, allowing the early boot code to perform a relocation to a different virtual address at runtime. This is a prerequisite for KASLR (CONFIG_RANDOMIZE_BASE). Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org> Signed-off-by: Catalin Marinas <catalin.marinas@arm.com> Bug: 30369029 Patchset: kaslr-arm64-4.4 (cherry picked from commit 1e48ef7fcc374051730381a2a05da77eb4eafdb0) Signed-off-by: Jeff Vander Stoep <jeffv@google.com> Change-Id: If02e065722d438f85feb62240fc230e16f58e912
* UPSTREAM: arm64: switch to relative exception tablesArd Biesheuvel2016-09-22
| | | | | | | | | | | | | | | | | | | | | | | | Instead of using absolute addresses for both the exception location and the fixup, use offsets relative to the exception table entry values. Not only does this cut the size of the exception table in half, it is also a prerequisite for KASLR, since absolute exception table entries are subject to dynamic relocation, which is incompatible with the sorting of the exception table that occurs at build time. This patch also introduces the _ASM_EXTABLE preprocessor macro (which exists on x86 as well) and its _asm_extable assembly counterpart, as shorthands to emit exception table entries. Acked-by: Will Deacon <will.deacon@arm.com> Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org> Signed-off-by: Catalin Marinas <catalin.marinas@arm.com> Bug: 30369029 Patchset: kaslr-arm64-4.4 (cherry picked from commit 6c94f27ac847ff8ef15b3da5b200574923bd6287) Signed-off-by: Jeff Vander Stoep <jeffv@google.com> Change-Id: Icedda8ee8c32843c439765783816d7d71ca0073a
* UPSTREAM: extable: add support for relative extables to search and sort routinesArd Biesheuvel2016-09-22
| | | | | | | | | | | | | | | | | | | | | | This adds support to the generic search_extable() and sort_extable() implementations for dealing with exception table entries whose fields contain relative offsets rather than absolute addresses. Acked-by: Helge Deller <deller@gmx.de> Acked-by: Heiko Carstens <heiko.carstens@de.ibm.com> Acked-by: H. Peter Anvin <hpa@linux.intel.com> Acked-by: Tony Luck <tony.luck@intel.com> Acked-by: Will Deacon <will.deacon@arm.com> Acked-by: Richard Henderson <rth@twiddle.net> Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org> Signed-off-by: Catalin Marinas <catalin.marinas@arm.com> Bug: 30369029 Patchset: kaslr-arm64-4.4 (cherry picked from commit a272858a3c1ecd4a935ba23c66668f81214bd110) Signed-off-by: Jeff Vander Stoep <jeffv@google.com> Change-Id: I9d144d351d547c49bf3203a69dfff3cb71a51177
* UPSTREAM: scripts/sortextable: add support for ET_DYN binariesArd Biesheuvel2016-09-22
| | | | | | | | | | | | | | | | Add support to scripts/sortextable for handling relocatable (PIE) executables, whose ELF type is ET_DYN, not ET_EXEC. Other than adding support for the new type, no changes are needed. Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org> Signed-off-by: Catalin Marinas <catalin.marinas@arm.com> Bug: 30369029 Patchset: kaslr-arm64-4.4 (cherry picked from commit 7b957b6e603623ef8b2e8222fa94b976df613fa2) Signed-off-by: Jeff Vander Stoep <jeffv@google.com> Change-Id: If55296ef4934b99c38ceb5acbd7c4a7fb23f24c1
* UPSTREAM: arm64: futex.h: Add missing PAN togglingJames Morse2016-09-22
| | | | | | | | | | | | | | | | | futex.h's futex_atomic_cmpxchg_inatomic() does not use the __futex_atomic_op() macro and needs its own PAN toggling. This was missed when the feature was implemented. Fixes: 338d4f49d6f ("arm64: kernel: Add support for Privileged Access Never") Signed-off-by: James Morse <james.morse@arm.com> Signed-off-by: Will Deacon <will.deacon@arm.com> Bug: 30369029 Patchset: kaslr-arm64-4.4 (cherry picked from commit 811d61e384e24759372bb3f01772f3744b0a8327) Signed-off-by: Jeff Vander Stoep <jeffv@google.com> Change-Id: I6e7b338a1af17b784d4196101422c3acee3b88ed
* UPSTREAM: arm64: make asm/elf.h available to asm filesArd Biesheuvel2016-09-22
| | | | | | | | | | | | | | | | | This reshuffles some code in asm/elf.h and puts a #ifndef __ASSEMBLY__ around its C definitions so that the CPP defines can be used in asm source files as well. Acked-by: Mark Rutland <mark.rutland@arm.com> Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org> Signed-off-by: Catalin Marinas <catalin.marinas@arm.com> Bug: 30369029 Patchset: kaslr-arm64-4.4 (cherry picked from commit 4a2e034e5cdadde4c712f79bdd57d1455c76a3db) Signed-off-by: Jeff Vander Stoep <jeffv@google.com> Change-Id: Ic499e950d2ef297d10848862a6dfa07b90887f4c
* UPSTREAM: arm64: avoid dynamic relocations in early boot codeArd Biesheuvel2016-09-22
| | | | | | | | | | | | | | | | | | | | | | | | Before implementing KASLR for arm64 by building a self-relocating PIE executable, we have to ensure that values we use before the relocation routine is executed are not subject to dynamic relocation themselves. This applies not only to virtual addresses, but also to values that are supplied by the linker at build time and relocated using R_AARCH64_ABS64 relocations. So instead, use assemble time constants, or force the use of static relocations by folding the constants into the instructions. Reviewed-by: Mark Rutland <mark.rutland@arm.com> Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org> Signed-off-by: Catalin Marinas <catalin.marinas@arm.com> Bug: 30369029 Patchset: kaslr-arm64-4.4 (cherry picked from commit 2bf31a4a05f5b00f37d65ba029d36a0230286cb7) Signed-off-by: Jeff Vander Stoep <jeffv@google.com> Change-Id: Icce0176591e3c0ae444e1ea54258efe677933c5b
* UPSTREAM: arm64: avoid R_AARCH64_ABS64 relocations for Image header fieldsArd Biesheuvel2016-09-22
| | | | | | | | | | | | | | | | | | | | | | Unfortunately, the current way of using the linker to emit build time constants into the Image header will no longer work once we switch to the use of PIE executables. The reason is that such constants are emitted into the binary using R_AARCH64_ABS64 relocations, which are resolved at runtime, not at build time, and the places targeted by those relocations will contain zeroes before that. So refactor the endian swapping linker script constant generation code so that it emits the upper and lower 32-bit words separately. Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org> Signed-off-by: Catalin Marinas <catalin.marinas@arm.com> Bug: 30369029 Patchset: kaslr-arm64-4.4 (cherry picked from commit 6ad1fe5d9077a1ab40bf74b61994d2e770b00b14) Signed-off-by: Jeff Vander Stoep <jeffv@google.com> Change-Id: Iaa809a0b5fcf628e1e49cd6aaa0f31f31ce95c23
* UPSTREAM: arm64: add support for module PLTsArd Biesheuvel2016-09-22
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This adds support for emitting PLTs at module load time for relative branches that are out of range. This is a prerequisite for KASLR, which may place the kernel and the modules anywhere in the vmalloc area, making it more likely that branch target offsets exceed the maximum range of +/- 128 MB. In this version, I removed the distinction between relocations against .init executable sections and ordinary executable sections. The reason is that it is hardly worth the trouble, given that .init.text usually does not contain that many far branches, and this version now only reserves PLT entry space for jump and call relocations against undefined symbols (since symbols defined in the same module can be assumed to be within +/- 128 MB) For example, the mac80211.ko module (which is fairly sizable at ~400 KB) built with -mcmodel=large gives the following relocation counts: relocs branches unique !local .text 3925 3347 518 219 .init.text 11 8 7 1 .exit.text 4 4 4 1 .text.unlikely 81 67 36 17 ('unique' means branches to unique type/symbol/addend combos, of which !local is the subset referring to undefined symbols) IOW, we are only emitting a single PLT entry for the .init sections, and we are better off just adding it to the core PLT section instead. Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org> Signed-off-by: Catalin Marinas <catalin.marinas@arm.com> Bug: 30369029 Patchset: kaslr-arm64-4.4 (cherry picked from commit fd045f6cd98ec4953147b318418bd45e441e52a3) Signed-off-by: Jeff Vander Stoep <jeffv@google.com> Change-Id: I1b46bb817e7d16a1b9a394b100c9e5de46c0837c
* UPSTREAM: arm64: move brk immediate argument definitions to separate headerArd Biesheuvel2016-09-22
| | | | | | | | | | | | | | | | | | | | Instead of reversing the header dependency between asm/bug.h and asm/debug-monitors.h, split off the brk instruction immediate value defines into a new header asm/brk-imm.h, and include it from both. This solves the circular dependency issue that prevents BUG() from being used in some header files, and keeps the definitions together. Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org> Acked-by: Will Deacon <will.deacon@arm.com> Signed-off-by: Catalin Marinas <catalin.marinas@arm.com> Bug: 30369029 Patchset: kaslr-arm64-4.4 (cherry picked from commit f98deee9a9f8c47d05a0f64d86440882dca772ff) Signed-off-by: Jeff Vander Stoep <jeffv@google.com> Change-Id: Id4827af98ab3d413828c589bc379acecabeff108
* UPSTREAM: arm64: mm: use bit ops rather than arithmetic in pa/va translationsArd Biesheuvel2016-09-22
| | | | | | | | | | | | | | | | | | | | | | Since PAGE_OFFSET is chosen such that it cuts the kernel VA space right in half, and since the size of the kernel VA space itself is always a power of 2, we can treat PAGE_OFFSET as a bitmask and replace the additions/subtractions with 'or' and 'and-not' operations. For the comparison against PAGE_OFFSET, a mov/cmp/branch sequence ends up getting replaced with a single tbz instruction. For the additions and subtractions, we save a mov instruction since the mask is folded into the instruction's immediate field. Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org> Signed-off-by: Catalin Marinas <catalin.marinas@arm.com> Bug: 30369029 Patchset: kaslr-arm64-4.4 (cherry picked from commit 8439e62a15614e8fcd43835d57b7245cd9870dc5) Signed-off-by: Jeff Vander Stoep <jeffv@google.com> Change-Id: I1ea4ef654dd7b7693f8713dab28ca0739b8a2c62
* UPSTREAM: arm64: mm: only perform memstart_addr sanity check if DEBUG_VMArd Biesheuvel2016-09-22
| | | | | | | | | | | | | | | | | | Checking whether memstart_addr has been assigned every time it is referenced adds a branch instruction that may hurt performance if the reference in question occurs on a hot path. So only perform the check if CONFIG_DEBUG_VM=y. Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org> [catalin.marinas@arm.com: replaced #ifdef with VM_BUG_ON] Signed-off-by: Catalin Marinas <catalin.marinas@arm.com> Bug: 30369029 Patchset: kaslr-arm64-4.4 (cherry picked from commit a92405f082d43267575444a6927085e4c8a69e4e) Signed-off-by: Jeff Vander Stoep <jeffv@google.com> Change-Id: Ia5f206d9a2dbbdbfc3f05fe985d4eca309f0d889
* UPSTREAM: arm64: User die() instead of panic() in do_page_fault()Catalin Marinas2016-09-22
| | | | | | | | | | | | | | The former gives better error reporting on unhandled permission faults (introduced by the UAO patches). Signed-off-by: Catalin Marinas <catalin.marinas@arm.com> Bug: 30369029 Patchset: kaslr-arm64-4.4 (cherry picked from commit 70c8abc28762d04e36c92e07eee2ce6ab41049cb) Signed-off-by: Jeff Vander Stoep <jeffv@google.com> Change-Id: Ia419eccf1554a32fa4131ac15b277d4d2d4eb508
* UPSTREAM: arm64: allow kernel Image to be loaded anywhere in physical memoryArd Biesheuvel2016-09-22
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This relaxes the kernel Image placement requirements, so that it may be placed at any 2 MB aligned offset in physical memory. This is accomplished by ignoring PHYS_OFFSET when installing memblocks, and accounting for the apparent virtual offset of the kernel Image. As a result, virtual address references below PAGE_OFFSET are correctly mapped onto physical references into the kernel Image regardless of where it sits in memory. Special care needs to be taken for dealing with memory limits passed via mem=, since the generic implementation clips memory top down, which may clip the kernel image itself if it is loaded high up in memory. To deal with this case, we simply add back the memory covering the kernel image, which may result in more memory to be retained than was passed as a mem= parameter. Since mem= should not be considered a production feature, a panic notifier handler is installed that dumps the memory limit at panic time if one was set. Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org> Signed-off-by: Catalin Marinas <catalin.marinas@arm.com> Bug: 30369029 Patchset: kaslr-arm64-4.4 (cherry picked from commit a7f8de168ace487fa7b88cb154e413cf40e87fc6) Signed-off-by: Jeff Vander Stoep <jeffv@google.com> Change-Id: I1d28cb66b658ef89f9648918565ddc07df4660f8
* UPSTREAM: arm64: defer __va translation of initrd_start and initrd_endArd Biesheuvel2016-09-22
| | | | | | | | | | | | | | | | | | | | | | | | Before deferring the assignment of memstart_addr in a subsequent patch, to the moment where all memory has been discovered and possibly clipped based on the size of the linear region and the presence of a mem= command line parameter, we need to ensure that memstart_addr is not used to perform __va translations before it is assigned. One such use is in the generic early DT discovery of the initrd location, which is recorded as a virtual address in the globals initrd_start and initrd_end. So wire up the generic support to declare the initrd addresses, and implement it without __va() translations, and perform the translation after memstart_addr has been assigned. Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org> Signed-off-by: Catalin Marinas <catalin.marinas@arm.com> Bug: 30369029 Patchset: kaslr-arm64-4.4 (cherry picked from commit a89dea585371a9d5d85499db47c93f129be8e0c4) Signed-off-by: Jeff Vander Stoep <jeffv@google.com> Change-Id: I7d0b3dd7adcf069d4e7c1f58fd12e59c4cb62017
* UPSTREAM: arm64: move kernel image to base of vmalloc areaArd Biesheuvel2016-09-22
| | | | | | | | | | | | | | | | | | | | | | | This moves the module area to right before the vmalloc area, and moves the kernel image to the base of the vmalloc area. This is an intermediate step towards implementing KASLR, which allows the kernel image to be located anywhere in the vmalloc area. Since other subsystems such as hibernate may still need to refer to the kernel text or data segments via their linears addresses, both are mapped in the linear region as well. The linear alias of the text region is mapped read-only/non-executable to prevent inadvertent modification or execution. Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org> Signed-off-by: Catalin Marinas <catalin.marinas@arm.com> Bug: 30369029 Patchset: kaslr-arm64-4.4 (cherry picked from commit f9040773b7bbbd9e98eb6184a263512a7cfc133f) Signed-off-by: Jeff Vander Stoep <jeffv@google.com> Change-Id: I698faed47bb7cfc256a1b5b5407a7c586bdc63b3
* BACKPORT: arm64: kvm: deal with kernel symbols outside of linear mappingArd Biesheuvel2016-09-22
| | | | | | | | | | | | | | | | | | | | KVM on arm64 uses a fixed offset between the linear mapping at EL1 and the HYP mapping at EL2. Before we can move the kernel virtual mapping out of the linear mapping, we have to make sure that references to kernel symbols that are accessed via the HYP mapping are translated to their linear equivalent. Reviewed-by: Mark Rutland <mark.rutland@arm.com> Acked-by: Marc Zyngier <marc.zyngier@arm.com> Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org> Signed-off-by: Catalin Marinas <catalin.marinas@arm.com> Bug: 30369029 Patchset: kaslr-arm64-4.4 (cherry picked from commit a0bf9776cd0be4490d4675d4108e13379849fc7f) Signed-off-by: Jeff Vander Stoep <jeffv@google.com> Change-Id: I316f029d22a16773c168a151dba59bed7921fa7e
* UPSTREAM: arm64: decouple early fixmap init from linear mappingArd Biesheuvel2016-09-22
| | | | | | | | | | | | | | | | | | | | | | | Since the early fixmap page tables are populated using pages that are part of the static footprint of the kernel, they are covered by the initial kernel mapping, and we can refer to them without using __va/__pa translations, which are tied to the linear mapping. Since the fixmap page tables are disjoint from the kernel mapping up to the top level pgd entry, we can refer to bm_pte[] directly, and there is no need to walk the page tables and perform __pa()/__va() translations at each step. Reviewed-by: Mark Rutland <mark.rutland@arm.com> Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org> Signed-off-by: Catalin Marinas <catalin.marinas@arm.com> Bug: 30369029 Patchset: kaslr-arm64-4.4 (cherry picked from commit 157962f5a8f236cab898b68bdaa69ce68922f0bf) Signed-off-by: Jeff Vander Stoep <jeffv@google.com> Change-Id: I49221a199962aec6d4f3712bfb3dd041d64ba99b
* UPSTREAM: arm64: pgtable: implement static [pte|pmd|pud]_offset variantsArd Biesheuvel2016-09-22
| | | | | | | | | | | | | | | | | | | | | The page table accessors pte_offset(), pud_offset() and pmd_offset() rely on __va translations, so they can only be used after the linear mapping has been installed. For the early fixmap and kasan init routines, whose page tables are allocated statically in the kernel image, these functions will return bogus values. So implement pte_offset_kimg(), pmd_offset_kimg() and pud_offset_kimg(), which can be used instead before any page tables have been allocated dynamically. Reviewed-by: Mark Rutland <mark.rutland@arm.com> Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org> Signed-off-by: Catalin Marinas <catalin.marinas@arm.com> Bug: 30369029 Patchset: kaslr-arm64-4.4 (cherry picked from commit 6533945a32c762c5db70d7a3ec251a040b2d9661) Signed-off-by: Jeff Vander Stoep <jeffv@google.com> Change-Id: Ibea400f0938db568524fb83eb2d22d8658bbb56b
* UPSTREAM: arm64: introduce KIMAGE_VADDR as the virtual base of the kernel regionArd Biesheuvel2016-09-22
| | | | | | | | | | | | | | | | | | | This introduces the preprocessor symbol KIMAGE_VADDR which will serve as the symbolic virtual base of the kernel region, i.e., the kernel's virtual offset will be KIMAGE_VADDR + TEXT_OFFSET. For now, we define it as being equal to PAGE_OFFSET, but in the future, it will be moved below it once we move the kernel virtual mapping out of the linear mapping. Reviewed-by: Mark Rutland <mark.rutland@arm.com> Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org> Signed-off-by: Catalin Marinas <catalin.marinas@arm.com> Bug: 30369029 Patchset: kaslr-arm64-4.4 (cherry picked from commit ab893fb9f1b17f02139bce547bb4b69e96b9ae16) Signed-off-by: Jeff Vander Stoep <jeffv@google.com> Change-Id: I31427bd2b948a22bb8ce1d22109682fc66efb98d
* BACKPORT: arm64: add support for ioremap() block mappingsArd Biesheuvel2016-09-22
| | | | | | | | | | | | | | | | | | This wires up the existing generic huge-vmap feature, which allows ioremap() to use PMD or PUD sized block mappings. It also adds support to the unmap path for dealing with block mappings, which will allow us to unmap the __init region using unmap_kernel_range() in a subsequent patch. Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org> Signed-off-by: Catalin Marinas <catalin.marinas@arm.com> Bug: 30369029 Patchset: kaslr-arm64-4.4 (cherry picked from commit 324420bf91f60582bb481133db9547111768ef17) Signed-off-by: Jeff Vander Stoep <jeffv@google.com> Change-Id: I4765ae77f7d67c3972b7e5b19d43db434e8b777c
* BACKPORT: arm64: prevent potential circular header dependencies in asm/bug.hArd Biesheuvel2016-09-22
| | | | | | | | | | | | | | | | | | | | Currently, using BUG_ON() in header files is cumbersome, due to the fact that asm/bug.h transitively includes a lot of other header files, resulting in the actual BUG_ON() invocation appearing before its definition in the preprocessor input. So let's reverse the #include dependency between asm/bug.h and asm/debug-monitors.h, by moving the definition of BUG_BRK_IMM from the latter to the former. Also fix up one user of asm/debug-monitors.h which relied on a transitive include. Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org> Signed-off-by: Catalin Marinas <catalin.marinas@arm.com> Bug: 30369029 Patchset: kaslr-arm64-4.4 (cherry picked from commit 03336b1df9929e5d9c28fd9768948b6151cb046c) Signed-off-by: Jeff Vander Stoep <jeffv@google.com> Change-Id: I71470b7db9ef858a5a8368a872f931936c723a25
* UPSTREAM: of/fdt: factor out assignment of initrd_start/initrd_endArd Biesheuvel2016-09-22
| | | | | | | | | | | | | | | | | | Since architectures may not yet have their linear mapping up and running when the initrd address is discovered from the DT, factor out the assignment of initrd_start and initrd_end, so that an architecture can override it and use the translation it needs. Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org> Acked-by: Rob Herring <robh@kernel.org> Signed-off-by: Catalin Marinas <catalin.marinas@arm.com> Bug: 30369029 Patchset: kaslr-arm64-4.4 (cherry picked from commit 369bc9abf22bf026e8645a4dd746b90649a2f6ee) Signed-off-by: Jeff Vander Stoep <jeffv@google.com> Change-Id: I8c258bdc4367955314e9a5223dc4c7751a06a98d
* UPSTREAM: of/fdt: make memblock minimum physical address arch configurableArd Biesheuvel2016-09-22
| | | | | | | | | | | | | | | | | | | | | | By default, early_init_dt_add_memory_arch() ignores memory below the base of the kernel image since it won't be addressable via the linear mapping. However, this is not appropriate anymore once we decouple the kernel text mapping from the linear mapping, so archs may want to drop the low limit entirely. So allow the minimum to be overridden by setting MIN_MEMBLOCK_ADDR. Acked-by: Mark Rutland <mark.rutland@arm.com> Acked-by: Rob Herring <robh@kernel.org> Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org> Signed-off-by: Catalin Marinas <catalin.marinas@arm.com> Bug: 30369029 Patchset: kaslr-arm64-4.4 (cherry picked from commit 270522a04f7a9911983878fa37da467f9ff1c938) Signed-off-by: Jeff Vander Stoep <jeffv@google.com> Change-Id: I4bb2626a87493262a64584b3d808de260129127e
* UPSTREAM: arm64: Remove the get_thread_info() functionCatalin Marinas2016-09-22
| | | | | | | | | | | | | | | | | This function was introduced by previous commits implementing UAO. However, it can be replaced with task_thread_info() in uao_thread_switch() or get_fs() in do_page_fault() (the latter being called only on the current context, so no need for using the saved pt_regs). Signed-off-by: Catalin Marinas <catalin.marinas@arm.com> Bug: 30369029 Patchset: kaslr-arm64-4.4 (cherry picked from commit e950631e84e7e38892ffbeee5e1816b270026b0e) Signed-off-by: Jeff Vander Stoep <jeffv@google.com> Change-Id: Ic6e9b6af7314fa83d9b0773ae3fac5a2ff34e67a
* BACKPORT: arm64: kernel: Don't toggle PAN on systems with UAOJames Morse2016-09-22
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | If a CPU supports both Privileged Access Never (PAN) and User Access Override (UAO), we don't need to disable/re-enable PAN round all copy_to_user() like calls. UAO alternatives cause these calls to use the 'unprivileged' load/store instructions, which are overridden to be the privileged kind when fs==KERNEL_DS. This patch changes the copy_to_user() calls to have their PAN toggling depend on a new composite 'feature' ARM64_ALT_PAN_NOT_UAO. If both features are detected, PAN will be enabled, but the copy_to_user() alternatives will not be applied. This means PAN will be enabled all the time for these functions. If only PAN is detected, the toggling will be enabled as normal. This will save the time taken to disable/re-enable PAN, and allow us to catch copy_to_user() accesses that occur with fs==KERNEL_DS. Futex and swp-emulation code continue to hang their PAN toggling code on ARM64_HAS_PAN. Signed-off-by: James Morse <james.morse@arm.com> Signed-off-by: Catalin Marinas <catalin.marinas@arm.com> Bug: 30369029 Patchset: kaslr-arm64-4.4 (cherry picked from commit 705441960033e66b63524521f153fbb28c99ddbd) Signed-off-by: Jeff Vander Stoep <jeffv@google.com> Change-Id: I3fa35ebacaf401e1344e76932a26fdd14a8a3cdb
* UPSTREAM: arm64: cpufeature: Test 'matches' pointer to find the end of the listJames Morse2016-09-22
| | | | | | | | | | | | | | | | | | | | | | | | | | | CPU feature code uses the desc field as a test to find the end of the list, this means every entry must have a description. This generates noise for entries in the list that aren't really features, but combinations of them. e.g. > CPU features: detected feature: Privileged Access Never > CPU features: detected feature: PAN and not UAO These combination features are needed for corner cases with alternatives, where cpu features interact. Change all walkers of the arm64_features[] and arm64_hwcaps[] lists to test 'matches' not 'desc', and only print 'desc' if it is non-NULL. Signed-off-by: James Morse <james.morse@arm.com> Reviewed-by : Suzuki K Poulose <suzuki.poulose@arm.com> Signed-off-by: Catalin Marinas <catalin.marinas@arm.com> Bug: 30369029 Patchset: kaslr-arm64-4.4 (cherry picked from commit 644c2ae198412c956700e55a2acf80b2541f6aa5) Signed-off-by: Jeff Vander Stoep <jeffv@google.com> Change-Id: I4500bb7c547e2e67ea56e242a8621df539f6fd67
* UPSTREAM: arm64: kernel: Add support for User Access OverrideJames Morse2016-09-22
| | | | | | | | | | | | | | | | | | | | | | | | | | 'User Access Override' is a new ARMv8.2 feature which allows the unprivileged load and store instructions to be overridden to behave in the normal way. This patch converts {get,put}_user() and friends to use ldtr*/sttr* instructions - so that they can only access EL0 memory, then enables UAO when fs==KERNEL_DS so that these functions can access kernel memory. This allows user space's read/write permissions to be checked against the page tables, instead of testing addr<USER_DS, then using the kernel's read/write permissions. Signed-off-by: James Morse <james.morse@arm.com> [catalin.marinas@arm.com: move uao_thread_switch() above dsb()] Signed-off-by: Catalin Marinas <catalin.marinas@arm.com> Bug: 30369029 Patchset: kaslr-arm64-4.4 (cherry picked from commit 57f4959bad0a154aeca125b7d38d1d9471a12422) Signed-off-by: Jeff Vander Stoep <jeffv@google.com> Change-Id: I1a6a74a1f33b92d54368bd99387b55cf62930903
* UPSTREAM: arm64: add ARMv8.2 id_aa64mmfr2 boiler plateJames Morse2016-09-22
| | | | | | | | | | | | | | | | | ARMv8.2 adds a new feature register id_aa64mmfr2. This patch adds the cpu feature boiler plate used by the actual features in later patches. Signed-off-by: James Morse <james.morse@arm.com> Reviewed-by: Suzuki K Poulose <suzuki.poulose@arm.com> Signed-off-by: Catalin Marinas <catalin.marinas@arm.com> Bug: 30369029 Patchset: kaslr-arm64-4.4 (cherry picked from commit 406e308770a92bd33995b2e5b681e86358328bb0) Signed-off-by: Jeff Vander Stoep <jeffv@google.com> Change-Id: I51db326696ba1e18e9a4c667acbeb3e25e0b151e
* UPSTREAM: arm64: cpufeature: Change read_cpuid() to use sysreg's mrs_s macroJames Morse2016-09-22
| | | | | | | | | | | | | | | | | | | | | | Older assemblers may not have support for newer feature registers. To get round this, sysreg.h provides a 'mrs_s' macro that takes a register encoding and generates the raw instruction. Change read_cpuid() to use mrs_s in all cases so that new registers don't have to be a special case. Including sysreg.h means we need to move the include and definition of read_cpuid() after the #ifndef __ASSEMBLY__ to avoid syntax errors in vmlinux.lds. Signed-off-by: James Morse <james.morse@arm.com> Acked-by: Mark Rutland <mark.rutland@arm.com> Signed-off-by: Catalin Marinas <catalin.marinas@arm.com> Bug: 30369029 Patchset: kaslr-arm64-4.4 (cherry picked from commit 0f54b14e76f5302afe164dc911b049b5df836ff5) Signed-off-by: Jeff Vander Stoep <jeffv@google.com> Change-Id: Ic59c81a97b4585b3c0964f293ddee08f7cb594ac
* UPSTREAM: arm64: use local label prefixes for __reg_num symbolsArd Biesheuvel2016-09-22
| | | | | | | | | | | | | | | | | | | | | | | | | | | The __reg_num_xNN symbols that are used to implement the msr_s and mrs_s macros are recorded in the ELF metadata of each object file. This does not affect the size of the final binary, but it does clutter the output of tools like readelf, i.e., $ readelf -a vmlinux |grep -c __reg_num_x 50976 So let's use symbols with the .L prefix, these are strictly local, and don't end up in the object files. $ readelf -a vmlinux |grep -c __reg_num_x 0 Acked-by: Will Deacon <will.deacon@arm.com> Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org> Signed-off-by: Catalin Marinas <catalin.marinas@arm.com> Bug: 30369029 Patchset: kaslr-arm64-4.4 (cherry picked from commit 7abc7d833c9eb16efc8a59239d3771a6e30be367) Signed-off-by: Jeff Vander Stoep <jeffv@google.com> Change-Id: Idefe9841ef7d1ddcc5161fc8de14153cfadaf4f3
* UPSTREAM: arm64: vdso: Mark vDSO code as read-onlyDavid Brown2016-09-22
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Although the arm64 vDSO is cleanly separated by code/data with the code being read-only in userspace mappings, the code page is still writable from the kernel. There have been exploits (such as http://itszn.com/blog/?p=21) that take advantage of this on x86 to go from a bad kernel write to full root. Prevent this specific exploit on arm64 by putting the vDSO code page in read-only memory as well. Before the change: [ 3.138366] vdso: 2 pages (1 code @ ffffffc000a71000, 1 data @ ffffffc000a70000) ---[ Kernel Mapping ]--- 0xffffffc000000000-0xffffffc000082000 520K RW NX SHD AF UXN MEM/NORMAL 0xffffffc000082000-0xffffffc000200000 1528K ro x SHD AF UXN MEM/NORMAL 0xffffffc000200000-0xffffffc000800000 6M ro x SHD AF BLK UXN MEM/NORMAL 0xffffffc000800000-0xffffffc0009b6000 1752K ro x SHD AF UXN MEM/NORMAL 0xffffffc0009b6000-0xffffffc000c00000 2344K RW NX SHD AF UXN MEM/NORMAL 0xffffffc000c00000-0xffffffc008000000 116M RW NX SHD AF BLK UXN MEM/NORMAL 0xffffffc00c000000-0xffffffc07f000000 1840M RW NX SHD AF BLK UXN MEM/NORMAL 0xffffffc800000000-0xffffffc840000000 1G RW NX SHD AF BLK UXN MEM/NORMAL 0xffffffc840000000-0xffffffc87ae00000 942M RW NX SHD AF BLK UXN MEM/NORMAL 0xffffffc87ae00000-0xffffffc87ae70000 448K RW NX SHD AF UXN MEM/NORMAL 0xffffffc87af80000-0xffffffc87af8a000 40K RW NX SHD AF UXN MEM/NORMAL 0xffffffc87af8b000-0xffffffc87b000000 468K RW NX SHD AF UXN MEM/NORMAL 0xffffffc87b000000-0xffffffc87fe00000 78M RW NX SHD AF BLK UXN MEM/NORMAL 0xffffffc87fe00000-0xffffffc87ff50000 1344K RW NX SHD AF UXN MEM/NORMAL 0xffffffc87ff90000-0xffffffc87ffa0000 64K RW NX SHD AF UXN MEM/NORMAL 0xffffffc87fff0000-0xffffffc880000000 64K RW NX SHD AF UXN MEM/NORMAL After: [ 3.138368] vdso: 2 pages (1 code @ ffffffc0006de000, 1 data @ ffffffc000a74000) ---[ Kernel Mapping ]--- 0xffffffc000000000-0xffffffc000082000 520K RW NX SHD AF UXN MEM/NORMAL 0xffffffc000082000-0xffffffc000200000 1528K ro x SHD AF UXN MEM/NORMAL 0xffffffc000200000-0xffffffc000800000 6M ro x SHD AF BLK UXN MEM/NORMAL 0xffffffc000800000-0xffffffc0009b8000 1760K ro x SHD AF UXN MEM/NORMAL 0xffffffc0009b8000-0xffffffc000c00000 2336K RW NX SHD AF UXN MEM/NORMAL 0xffffffc000c00000-0xffffffc008000000 116M RW NX SHD AF BLK UXN MEM/NORMAL 0xffffffc00c000000-0xffffffc07f000000 1840M RW NX SHD AF BLK UXN MEM/NORMAL 0xffffffc800000000-0xffffffc840000000 1G RW NX SHD AF BLK UXN MEM/NORMAL 0xffffffc840000000-0xffffffc87ae00000 942M RW NX SHD AF BLK UXN MEM/NORMAL 0xffffffc87ae00000-0xffffffc87ae70000 448K RW NX SHD AF UXN MEM/NORMAL 0xffffffc87af80000-0xffffffc87af8a000 40K RW NX SHD AF UXN MEM/NORMAL 0xffffffc87af8b000-0xffffffc87b000000 468K RW NX SHD AF UXN MEM/NORMAL 0xffffffc87b000000-0xffffffc87fe00000 78M RW NX SHD AF BLK UXN MEM/NORMAL 0xffffffc87fe00000-0xffffffc87ff50000 1344K RW NX SHD AF UXN MEM/NORMAL 0xffffffc87ff90000-0xffffffc87ffa0000 64K RW NX SHD AF UXN MEM/NORMAL 0xffffffc87fff0000-0xffffffc880000000 64K RW NX SHD AF UXN MEM/NORMAL Inspired by https://lkml.org/lkml/2016/1/19/494 based on work by the PaX Team, Brad Spengler, and Kees Cook. Signed-off-by: David Brown <david.brown@linaro.org> Acked-by: Will Deacon <will.deacon@arm.com> Acked-by: Ard Biesheuvel <ard.biesheuvel@linaro.org> [catalin.marinas@arm.com: removed superfluous __PAGE_ALIGNED_DATA] Signed-off-by: Catalin Marinas <catalin.marinas@arm.com> Bug: 30369029 Patchset: kaslr-arm64-4.4 (cherry picked from commit 88d8a7994e564d209d4b2583496631c2357d386b) Signed-off-by: Jeff Vander Stoep <jeffv@google.com> Change-Id: I3fe4b48df8b27313ac61c947746805442757932c
generated by cgit v1.2.3 (git 2.25.1) at 2025-12-16 03:22:26 +0000