Add Intel PCID plumbing |
|||||||||||||||
Issue descriptionWill be used later.
,
Oct 27 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromiumos/third_party/kernel/+/aaad659fe4c860ec2d2c03eaf45c191d5078c8a3 commit aaad659fe4c860ec2d2c03eaf45c191d5078c8a3 Author: Borislav Petkov <bp@suse.de> Date: Fri Oct 27 23:12:32 2017 UPSTREAM: x86/mm: Fix INVPCID asm constraint So we want to specify the dependency on both @pcid and @addr so that the compiler doesn't reorder accesses to them *before* the TLB flush. But for that to work, we need to express this properly in the inline asm and deref the whole desc array, not the pointer to it. See clwb() for an example. This fixes the build error on 32-bit: arch/x86/include/asm/tlbflush.h: In function __invpcid: arch/x86/include/asm/tlbflush.h:26:18: error: memory input 0 is not directly addressable which gcc4.7 caught but 5.x didn't. Which is strange. :-\ BUG=chromium:779077 TEST=Build, boot on caroline. Signed-off-by: Borislav Petkov <bp@suse.de> Cc: Andrew Morton <akpm@linux-foundation.org> Cc: Andrey Ryabinin <aryabinin@virtuozzo.com> Cc: Andy Lutomirski <luto@amacapital.net> Cc: Borislav Petkov <bp@alien8.de> Cc: Brian Gerst <brgerst@gmail.com> Cc: Dave Hansen <dave.hansen@linux.intel.com> Cc: Denys Vlasenko <dvlasenk@redhat.com> Cc: H. Peter Anvin <hpa@zytor.com> Cc: Linus Torvalds <torvalds@linux-foundation.org> Cc: Luis R. Rodriguez <mcgrof@suse.com> Cc: Michael Matz <matz@suse.de> Cc: Oleg Nesterov <oleg@redhat.com> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Thomas Gleixner <tglx@linutronix.de> Cc: Toshi Kani <toshi.kani@hp.com> Cc: linux-mm@kvack.org Signed-off-by: Ingo Molnar <mingo@kernel.org> (cherry picked from commit e2c7698cd61f11d4077fdb28148b2d31b82ac848) Change-Id: I428b1d1445f6756b7644869a5d323ef041799339 Reviewed-on: https://chromium-review.googlesource.com/742003 Commit-Ready: Jorge Lucangeli Obes <jorgelo@chromium.org> Tested-by: Jorge Lucangeli Obes <jorgelo@chromium.org> Reviewed-by: Guenter Roeck <groeck@chromium.org> [modify] https://crrev.com/aaad659fe4c860ec2d2c03eaf45c191d5078c8a3/arch/x86/include/asm/tlbflush.h
,
Oct 27 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromiumos/third_party/kernel/+/bfc547bba1aec1fd5e7fd88de6a91523b727cca3 commit bfc547bba1aec1fd5e7fd88de6a91523b727cca3 Author: Andy Lutomirski <luto@kernel.org> Date: Fri Oct 27 23:12:33 2017 UPSTREAM: x86/mm: Add a 'noinvpcid' boot option to turn off INVPCID This adds a chicken bit to turn off INVPCID in case something goes wrong. It's an early_param() because we do TLB flushes before we parse __setup() parameters. BUG=chromium:779077 TEST=Build, boot on caroline. Signed-off-by: Andy Lutomirski <luto@kernel.org> Reviewed-by: Borislav Petkov <bp@suse.de> Cc: Andrew Morton <akpm@linux-foundation.org> Cc: Andrey Ryabinin <aryabinin@virtuozzo.com> Cc: Andy Lutomirski <luto@amacapital.net> Cc: Borislav Petkov <bp@alien8.de> Cc: Brian Gerst <brgerst@gmail.com> Cc: Dave Hansen <dave.hansen@linux.intel.com> Cc: Denys Vlasenko <dvlasenk@redhat.com> Cc: H. Peter Anvin <hpa@zytor.com> Cc: Linus Torvalds <torvalds@linux-foundation.org> Cc: Luis R. Rodriguez <mcgrof@suse.com> Cc: Oleg Nesterov <oleg@redhat.com> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Thomas Gleixner <tglx@linutronix.de> Cc: Toshi Kani <toshi.kani@hp.com> Cc: linux-mm@kvack.org Link: http://lkml.kernel.org/r/f586317ed1bc2b87aee652267e515b90051af385.1454096309.git.luto@kernel.org Signed-off-by: Ingo Molnar <mingo@kernel.org> (cherry picked from commit d12a72b844a49d4162f24cefdab30bed3f86730e) Change-Id: I259a8968ddcaca10edaa3fd7976a2a25d2afd4db Reviewed-on: https://chromium-review.googlesource.com/742004 Commit-Ready: Jorge Lucangeli Obes <jorgelo@chromium.org> Tested-by: Jorge Lucangeli Obes <jorgelo@chromium.org> Reviewed-by: Guenter Roeck <groeck@chromium.org> [modify] https://crrev.com/bfc547bba1aec1fd5e7fd88de6a91523b727cca3/Documentation/kernel-parameters.txt [modify] https://crrev.com/bfc547bba1aec1fd5e7fd88de6a91523b727cca3/arch/x86/kernel/cpu/common.c
,
Oct 27 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromiumos/third_party/kernel/+/57043bc514d010f5f616058e85b842a6b5245ef0 commit 57043bc514d010f5f616058e85b842a6b5245ef0 Author: Andy Lutomirski <luto@kernel.org> Date: Fri Oct 27 23:12:34 2017 UPSTREAM: x86/mm: If INVPCID is available, use it to flush global mappings On my Skylake laptop, INVPCID function 2 (flush absolutely everything) takes about 376ns, whereas saving flags, twiddling CR4.PGE to flush global mappings, and restoring flags takes about 539ns. BUG=chromium:779077 TEST=Build, boot on caroline. Signed-off-by: Andy Lutomirski <luto@kernel.org> Reviewed-by: Borislav Petkov <bp@suse.de> Cc: Andrew Morton <akpm@linux-foundation.org> Cc: Andrey Ryabinin <aryabinin@virtuozzo.com> Cc: Andy Lutomirski <luto@amacapital.net> Cc: Borislav Petkov <bp@alien8.de> Cc: Brian Gerst <brgerst@gmail.com> Cc: Dave Hansen <dave.hansen@linux.intel.com> Cc: Denys Vlasenko <dvlasenk@redhat.com> Cc: H. Peter Anvin <hpa@zytor.com> Cc: Linus Torvalds <torvalds@linux-foundation.org> Cc: Luis R. Rodriguez <mcgrof@suse.com> Cc: Oleg Nesterov <oleg@redhat.com> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Thomas Gleixner <tglx@linutronix.de> Cc: Toshi Kani <toshi.kani@hp.com> Cc: linux-mm@kvack.org Link: http://lkml.kernel.org/r/ed0ef62581c0ea9c99b9bf6df726015e96d44743.1454096309.git.luto@kernel.org Signed-off-by: Ingo Molnar <mingo@kernel.org> (cherry picked from commit d8bced79af1db6734f66b42064cc773cada2ce99) Change-Id: I0a79ba87173caa1de1ce03dfd9d98258344103f3 Reviewed-on: https://chromium-review.googlesource.com/742005 Commit-Ready: Jorge Lucangeli Obes <jorgelo@chromium.org> Tested-by: Jorge Lucangeli Obes <jorgelo@chromium.org> Reviewed-by: Guenter Roeck <groeck@chromium.org> [modify] https://crrev.com/57043bc514d010f5f616058e85b842a6b5245ef0/arch/x86/include/asm/tlbflush.h
,
Oct 27 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromiumos/third_party/kernel/+/7552a0684da22ce7858edf98fcf563eb326de0ee commit 7552a0684da22ce7858edf98fcf563eb326de0ee Author: Ingo Molnar <mingo@kernel.org> Date: Fri Oct 27 23:12:36 2017 UPSTREAM: mm/mmu_context, sched/core: Fix mmu_context.h assumption Some architectures (such as Alpha) rely on include/linux/sched.h definitions in their mmu_context.h files. So include sched.h before mmu_context.h. BUG=chromium:779077 TEST=Build, boot on caroline. Cc: Andy Lutomirski <luto@kernel.org> Cc: Borislav Petkov <bp@alien8.de> Cc: Linus Torvalds <torvalds@linux-foundation.org> Cc: linux-kernel@vger.kernel.org Cc: Peter Zijlstra <peterz@infradead.org> Cc: Thomas Gleixner <tglx@linutronix.de> Signed-off-by: Ingo Molnar <mingo@kernel.org> (cherry picked from commit 8efd755ac2fe262d4c8d5c9bbe054bb67dae93da) Change-Id: I2a4fcda74e6aa4576a899439fb35c065f86711b4 Reviewed-on: https://chromium-review.googlesource.com/742006 Commit-Ready: Jorge Lucangeli Obes <jorgelo@chromium.org> Tested-by: Jorge Lucangeli Obes <jorgelo@chromium.org> Reviewed-by: Guenter Roeck <groeck@chromium.org> [modify] https://crrev.com/7552a0684da22ce7858edf98fcf563eb326de0ee/mm/mmu_context.c
,
Oct 27 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromiumos/third_party/kernel/+/ecf090f8fa9f3a06a3e6c189bb92afdc52cfa3bd commit ecf090f8fa9f3a06a3e6c189bb92afdc52cfa3bd Author: Andy Lutomirski <luto@kernel.org> Date: Fri Oct 27 23:12:37 2017 UPSTREAM: sched/core: Add switch_mm_irqs_off() and use it in the scheduler By default, this is the same thing as switch_mm(). x86 will override it as an optimization. BUG=chromium:779077 TEST=Build, boot on caroline. Signed-off-by: Andy Lutomirski <luto@kernel.org> Reviewed-by: Borislav Petkov <bp@suse.de> Cc: Borislav Petkov <bp@alien8.de> Cc: Linus Torvalds <torvalds@linux-foundation.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Thomas Gleixner <tglx@linutronix.de> Link: http://lkml.kernel.org/r/df401df47bdd6be3e389c6f1e3f5310d70e81b2c.1461688545.git.luto@kernel.org Signed-off-by: Ingo Molnar <mingo@kernel.org> (cherry picked from commit f98db6013c557c216da5038d9c52045be55cd039) Change-Id: I73dbc3409895bce461674be4480f8285acc35728 Reviewed-on: https://chromium-review.googlesource.com/742007 Commit-Ready: Jorge Lucangeli Obes <jorgelo@chromium.org> Tested-by: Jorge Lucangeli Obes <jorgelo@chromium.org> Reviewed-by: Guenter Roeck <groeck@chromium.org> [modify] https://crrev.com/ecf090f8fa9f3a06a3e6c189bb92afdc52cfa3bd/include/linux/mmu_context.h [modify] https://crrev.com/ecf090f8fa9f3a06a3e6c189bb92afdc52cfa3bd/kernel/sched/core.c
,
Oct 27 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromiumos/third_party/kernel/+/71fd8acecb1d42535dd9d1ffbeaa1e6feab20fa9 commit 71fd8acecb1d42535dd9d1ffbeaa1e6feab20fa9 Author: Andy Lutomirski <luto@kernel.org> Date: Fri Oct 27 23:12:38 2017 BACKPORT: x86/mm: Build arch/x86/mm/tlb.c even on !SMP Currently all of the functions that live in tlb.c are inlined on !SMP builds. One can debate whether this is a good idea (in many respects the code in tlb.c is better than the inlined UP code). Regardless, I want to add code that needs to be built on UP and SMP kernels and relates to tlb flushing, so arrange for tlb.c to be compiled unconditionally. BUG=chromium:779077 TEST=Build, boot on caroline. Signed-off-by: Andy Lutomirski <luto@kernel.org> Reviewed-by: Borislav Petkov <bp@suse.de> Cc: Borislav Petkov <bp@alien8.de> Cc: Linus Torvalds <torvalds@linux-foundation.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Thomas Gleixner <tglx@linutronix.de> Link: http://lkml.kernel.org/r/f0d778f0d828fc46e5d1946bca80f0aaf9abf032.1461688545.git.luto@kernel.org Signed-off-by: Ingo Molnar <mingo@kernel.org> (cherry picked from commit e1074888c326038340a1ada9129d679e661f2ea6) Change-Id: I7649a2495ba7d47be2a379a086a574c1b6305aee Reviewed-on: https://chromium-review.googlesource.com/742008 Commit-Ready: Jorge Lucangeli Obes <jorgelo@chromium.org> Tested-by: Jorge Lucangeli Obes <jorgelo@chromium.org> Reviewed-by: Guenter Roeck <groeck@chromium.org> [modify] https://crrev.com/71fd8acecb1d42535dd9d1ffbeaa1e6feab20fa9/arch/x86/mm/tlb.c [modify] https://crrev.com/71fd8acecb1d42535dd9d1ffbeaa1e6feab20fa9/arch/x86/mm/Makefile
,
Oct 27 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromiumos/third_party/kernel/+/4e6dbe4156b20aa2c08d312d18789ff3f796e7e3 commit 4e6dbe4156b20aa2c08d312d18789ff3f796e7e3 Author: Andy Lutomirski <luto@kernel.org> Date: Fri Oct 27 23:12:39 2017 BACKPORT: x86/mm, sched/core: Uninline switch_mm() It's fairly large and it has quite a few callers. This may also help untangle some headers down the road. BUG=chromium:779077 TEST=Build, boot on caroline. Signed-off-by: Andy Lutomirski <luto@kernel.org> Reviewed-by: Borislav Petkov <bp@suse.de> Cc: Borislav Petkov <bp@alien8.de> Cc: Linus Torvalds <torvalds@linux-foundation.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Thomas Gleixner <tglx@linutronix.de> Link: http://lkml.kernel.org/r/54f3367803e7f80b2be62c8a21879aa74b1a5f57.1461688545.git.luto@kernel.org Signed-off-by: Ingo Molnar <mingo@kernel.org> (cherry picked from commit 69c0319aabba45bcf33178916a2f06967b4adede) Conflicts: arch/x86/include/asm/mmu_context.h Change-Id: I82a8ef651aabaaf9f9d7859a46d1fd24aca81a8c Reviewed-on: https://chromium-review.googlesource.com/742009 Commit-Ready: Jorge Lucangeli Obes <jorgelo@chromium.org> Tested-by: Jorge Lucangeli Obes <jorgelo@chromium.org> Reviewed-by: Guenter Roeck <groeck@chromium.org> [modify] https://crrev.com/4e6dbe4156b20aa2c08d312d18789ff3f796e7e3/arch/x86/include/asm/mmu_context.h [modify] https://crrev.com/4e6dbe4156b20aa2c08d312d18789ff3f796e7e3/arch/x86/mm/tlb.c
,
Oct 27 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromiumos/third_party/kernel/+/b41337070064e63e475627b40585f5c0adb9c60a commit b41337070064e63e475627b40585f5c0adb9c60a Author: Andy Lutomirski <luto@kernel.org> Date: Fri Oct 27 23:12:40 2017 UPSTREAM: x86/mm, sched/core: Turn off IRQs in switch_mm() Potential races between switch_mm() and TLB-flush or LDT-flush IPIs could be very messy. AFAICT the code is currently okay, whether by accident or by careful design, but enabling PCID will make it considerably more complicated and will no longer be obviously safe. Fix it with a big hammer: run switch_mm() with IRQs off. To avoid a performance hit in the scheduler, we take advantage of our knowledge that the scheduler already has IRQs disabled when it calls switch_mm(). BUG=chromium:779077 TEST=Build, boot on caroline. Signed-off-by: Andy Lutomirski <luto@kernel.org> Reviewed-by: Borislav Petkov <bp@suse.de> Cc: Borislav Petkov <bp@alien8.de> Cc: Linus Torvalds <torvalds@linux-foundation.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Thomas Gleixner <tglx@linutronix.de> Link: http://lkml.kernel.org/r/f19baf759693c9dcae64bbff76189db77cb13398.1461688545.git.luto@kernel.org Signed-off-by: Ingo Molnar <mingo@kernel.org> (cherry picked from commit 078194f8e9fe3cf54c8fd8bded48a1db5bd8eb8a) Change-Id: I2141b622df2cdb2bf09e4d7e0e8078d3c6434c2b Reviewed-on: https://chromium-review.googlesource.com/742010 Commit-Ready: Jorge Lucangeli Obes <jorgelo@chromium.org> Tested-by: Jorge Lucangeli Obes <jorgelo@chromium.org> Reviewed-by: Guenter Roeck <groeck@chromium.org> [modify] https://crrev.com/b41337070064e63e475627b40585f5c0adb9c60a/arch/x86/include/asm/mmu_context.h [modify] https://crrev.com/b41337070064e63e475627b40585f5c0adb9c60a/arch/x86/mm/tlb.c
,
Oct 27 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromiumos/third_party/kernel/+/7377fe85a61e783ae6e55cd1511aee10d4ee4558 commit 7377fe85a61e783ae6e55cd1511aee10d4ee4558 Author: Steven Rostedt <rostedt@goodmis.org> Date: Fri Oct 27 23:12:41 2017 UPSTREAM: ARM: Hide finish_arch_post_lock_switch() from modules The introduction of switch_mm_irqs_off() brought back an old bug regarding the use of preempt_enable_no_resched: As part of: 62b94a08da1b ("sched/preempt: Take away preempt_enable_no_resched() from modules") the definition of preempt_enable_no_resched() is only available in built-in code, not in loadable modules, so we can't generally use it from header files. However, the ARM version of finish_arch_post_lock_switch() calls preempt_enable_no_resched() and is defined as a static inline function in asm/mmu_context.h. This in turn means we cannot include asm/mmu_context.h from modules. With today's tip tree, asm/mmu_context.h gets included from linux/mmu_context.h, which is normally the exact pattern one would expect, but unfortunately, linux/mmu_context.h can be included from the vhost driver that is a loadable module, now causing this compile time error with modular configs: In file included from ../include/linux/mmu_context.h:4:0, from ../drivers/vhost/vhost.c:18: ../arch/arm/include/asm/mmu_context.h: In function 'finish_arch_post_lock_switch': ../arch/arm/include/asm/mmu_context.h:88:3: error: implicit declaration of function 'preempt_enable_no_resched' [-Werror=implicit-function-declaration] preempt_enable_no_resched(); Andy already tried to fix the bug by including linux/preempt.h from asm/mmu_context.h, but that didn't help. Arnd suggested reordering the header files, which wasn't popular, so let's use this workaround instead: The finish_arch_post_lock_switch() definition is now also hidden inside of #ifdef MODULE, so we don't see anything referencing preempt_enable_no_resched() from a header file. I've built a few hundred randconfig kernels with this, and did not see any new problems. BUG=chromium:779077 TEST=Build, boot on caroline. Tested-by: Guenter Roeck <linux@roeck-us.net> Signed-off-by: Steven Rostedt <rostedt@goodmis.org> Signed-off-by: Arnd Bergmann <arnd@arndb.de> Acked-by: Russell King <rmk+kernel@arm.linux.org.uk> Cc: Alexander Shishkin <alexander.shishkin@linux.intel.com> Cc: Andy Lutomirski <luto@amacapital.net> Cc: Andy Lutomirski <luto@kernel.org> Cc: Ard Biesheuvel <ard.biesheuvel@linaro.org> Cc: Arnaldo Carvalho de Melo <acme@redhat.com> Cc: Borislav Petkov <bp@suse.de> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Linus Torvalds <torvalds@linux-foundation.org> Cc: Mel Gorman <mgorman@techsingularity.net> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Russell King - ARM Linux <linux@armlinux.org.uk> Cc: Stephane Eranian <eranian@google.com> Cc: Thomas Gleixner <tglx@linutronix.de> Cc: Vince Weaver <vincent.weaver@maine.edu> Cc: linux-arm-kernel@lists.infradead.org Fixes: f98db6013c55 ("sched/core: Add switch_mm_irqs_off() and use it in the scheduler") Link: http://lkml.kernel.org/r/1463146234-161304-1-git-send-email-arnd@arndb.de Signed-off-by: Ingo Molnar <mingo@kernel.org> (cherry picked from commit ef0491ea17f8019821c7e9c8e801184ecf17f85a) Change-Id: I2d9b447238d9b06a2adc812e3139e7d77313c34a Reviewed-on: https://chromium-review.googlesource.com/742011 Commit-Ready: Jorge Lucangeli Obes <jorgelo@chromium.org> Tested-by: Jorge Lucangeli Obes <jorgelo@chromium.org> Reviewed-by: Guenter Roeck <groeck@chromium.org> [modify] https://crrev.com/7377fe85a61e783ae6e55cd1511aee10d4ee4558/arch/arm/include/asm/mmu_context.h
,
Oct 27 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromiumos/third_party/kernel/+/b7987f7fed11516c2566883f783b77f6fcd36ab5 commit b7987f7fed11516c2566883f783b77f6fcd36ab5 Author: Andy Lutomirski <luto@kernel.org> Date: Fri Oct 27 23:12:43 2017 UPSTREAM: sched/core: Idle_task_exit() shouldn't use switch_mm_irqs_off() idle_task_exit() can be called with IRQs on x86 on and therefore should use switch_mm(), not switch_mm_irqs_off(). This doesn't seem to cause any problems right now, but it will confuse my upcoming TLB flush changes. Nonetheless, I think it should be backported because it's trivial. There won't be any meaningful performance impact because idle_task_exit() is only used when offlining a CPU. BUG=chromium:779077 TEST=Build, boot on caroline. Signed-off-by: Andy Lutomirski <luto@kernel.org> Cc: Borislav Petkov <bp@suse.de> Cc: Linus Torvalds <torvalds@linux-foundation.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Thomas Gleixner <tglx@linutronix.de> Cc: stable@vger.kernel.org Fixes: f98db6013c55 ("sched/core: Add switch_mm_irqs_off() and use it in the scheduler") Link: http://lkml.kernel.org/r/ca3d1a9fa93a0b49f5a8ff729eda3640fb6abdf9.1497034141.git.luto@kernel.org Signed-off-by: Ingo Molnar <mingo@kernel.org> (cherry picked from commit 252d2a4117bc181b287eeddf848863788da733ae) Change-Id: Icf9b0f7b3fa6362e206ce8d6f8176d9271ba61b9 Reviewed-on: https://chromium-review.googlesource.com/742012 Commit-Ready: Jorge Lucangeli Obes <jorgelo@chromium.org> Tested-by: Jorge Lucangeli Obes <jorgelo@chromium.org> Reviewed-by: Guenter Roeck <groeck@chromium.org> [modify] https://crrev.com/b7987f7fed11516c2566883f783b77f6fcd36ab5/kernel/sched/core.c
,
Oct 27 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromiumos/third_party/kernel/+/de27d1fdb7e93422130e22e319a54b7346f9706a commit de27d1fdb7e93422130e22e319a54b7346f9706a Author: Sebastian Andrzej Siewior <bigeasy@linutronix.de> Date: Fri Oct 27 23:12:44 2017 UPSTREAM: x86/mm: Disable preemption during CR3 read+write There's a subtle preemption race on UP kernels: Usually current->mm (and therefore mm->pgd) stays the same during the lifetime of a task so it does not matter if a task gets preempted during the read and write of the CR3. But then, there is this scenario on x86-UP: TaskA is in do_exit() and exit_mm() sets current->mm = NULL followed by: -> mmput() -> exit_mmap() -> tlb_finish_mmu() -> tlb_flush_mmu() -> tlb_flush_mmu_tlbonly() -> tlb_flush() -> flush_tlb_mm_range() -> __flush_tlb_up() -> __flush_tlb() -> __native_flush_tlb() At this point current->mm is NULL but current->active_mm still points to the "old" mm. Let's preempt taskA _after_ native_read_cr3() by taskB. TaskB has its own mm so CR3 has changed. Now preempt back to taskA. TaskA has no ->mm set so it borrows taskB's mm and so CR3 remains unchanged. Once taskA gets active it continues where it was interrupted and that means it writes its old CR3 value back. Everything is fine because userland won't need its memory anymore. Now the fun part: Let's preempt taskA one more time and get back to taskB. This time switch_mm() won't do a thing because oldmm (->active_mm) is the same as mm (as per context_switch()). So we remain with a bad CR3 / PGD and return to userland. The next thing that happens is handle_mm_fault() with an address for the execution of its code in userland. handle_mm_fault() realizes that it has a PTE with proper rights so it returns doing nothing. But the CPU looks at the wrong PGD and insists that something is wrong and faults again. And again. And one more time This pagefault circle continues until the scheduler gets tired of it and puts another task on the CPU. It gets little difficult if the task is a RT task with a high priority. The system will either freeze or it gets fixed by the software watchdog thread which usually runs at RT-max prio. But waiting for the watchdog will increase the latency of the RT task which is no good. Fix this by disabling preemption across the critical code section. BUG=chromium:779077 TEST=Build, boot on caroline. Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de> Acked-by: Peter Zijlstra (Intel) <peterz@infradead.org> Acked-by: Rik van Riel <riel@redhat.com> Acked-by: Andy Lutomirski <luto@kernel.org> Cc: Borislav Petkov <bp@alien8.de> Cc: Borislav Petkov <bp@suse.de> Cc: Brian Gerst <brgerst@gmail.com> Cc: Denys Vlasenko <dvlasenk@redhat.com> Cc: H. Peter Anvin <hpa@zytor.com> Cc: Josh Poimboeuf <jpoimboe@redhat.com> Cc: Linus Torvalds <torvalds@linux-foundation.org> Cc: Mel Gorman <mgorman@suse.de> Cc: Peter Zijlstra <a.p.zijlstra@chello.nl> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Thomas Gleixner <tglx@linutronix.de> Cc: linux-mm@kvack.org Cc: stable@vger.kernel.org Link: http://lkml.kernel.org/r/1470404259-26290-1-git-send-email-bigeasy@linutronix.de [ Prettified the changelog. ] Signed-off-by: Ingo Molnar <mingo@kernel.org> (cherry picked from commit 5cf0791da5c162ebc14b01eb01631cfa7ed4fa6e) Change-Id: Idfbcc9265368751dcbbb5e91fc35ec2b60d9a6c4 Reviewed-on: https://chromium-review.googlesource.com/742013 Commit-Ready: Jorge Lucangeli Obes <jorgelo@chromium.org> Tested-by: Jorge Lucangeli Obes <jorgelo@chromium.org> Reviewed-by: Guenter Roeck <groeck@chromium.org> [modify] https://crrev.com/de27d1fdb7e93422130e22e319a54b7346f9706a/arch/x86/include/asm/tlbflush.h
,
Oct 27 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromiumos/third_party/kernel/+/14eed2a7509e56d48a3ac7b6b33cf2375840bc98 commit 14eed2a7509e56d48a3ac7b6b33cf2375840bc98 Author: Aaron Lu <aaron.lu@intel.com> Date: Fri Oct 27 23:12:45 2017 UPSTREAM: x86/irq: Do not substract irq_tlb_count from irq_call_count Since commit: 52aec3308db8 ("x86/tlb: replace INVALIDATE_TLB_VECTOR by CALL_FUNCTION_VECTOR") the TLB remote shootdown is done through call function vector. That commit didn't take care of irq_tlb_count, which a later commit: fd0f5869724f ("x86: Distinguish TLB shootdown interrupts from other functions call interrupts") ... tried to fix. The fix assumes every increase of irq_tlb_count has a corresponding increase of irq_call_count. So the irq_call_count is always bigger than irq_tlb_count and we could substract irq_tlb_count from irq_call_count. Unfortunately this is not true for the smp_call_function_single() case. The IPI is only sent if the target CPU's call_single_queue is empty when adding a csd into it in generic_exec_single. That means if two threads are both adding flush tlb csds to the same CPU's call_single_queue, only one IPI is sent. In other words, the irq_call_count is incremented by 1 but irq_tlb_count is incremented by 2. Over time, irq_tlb_count will be bigger than irq_call_count and the substract will produce a very large irq_call_count value due to overflow. Considering that: 1) it's not worth to send more IPIs for the sake of accurate counting of irq_call_count in generic_exec_single(); 2) it's not easy to tell if the call function interrupt is for TLB shootdown in __smp_call_function_single_interrupt(). Not to exclude TLB shootdown from call function count seems to be the simplest fix and this patch just does that. This bug was found by LKP's cyclic performance regression tracking recently with the vm-scalability test suite. I have bisected to commit: 3dec0ba0be6a ("mm/rmap: share the i_mmap_rwsem") This commit didn't do anything wrong but revealed the irq_call_count problem. IIUC, the commit makes rwc->remap_one in rmap_walk_file concurrent with multiple threads. When remap_one is try_to_unmap_one(), then multiple threads could queue flush TLB to the same CPU but only one IPI will be sent. Since the commit was added in Linux v3.19, the counting problem only shows up from v3.19 onwards. BUG=chromium:779077 TEST=Build, boot on caroline. Signed-off-by: Aaron Lu <aaron.lu@intel.com> Cc: Alex Shi <alex.shi@linaro.org> Cc: Andy Lutomirski <luto@kernel.org> Cc: Borislav Petkov <bp@alien8.de> Cc: Brian Gerst <brgerst@gmail.com> Cc: Davidlohr Bueso <dave@stgolabs.net> Cc: Denys Vlasenko <dvlasenk@redhat.com> Cc: H. Peter Anvin <hpa@zytor.com> Cc: Huang Ying <ying.huang@intel.com> Cc: Josh Poimboeuf <jpoimboe@redhat.com> Cc: Linus Torvalds <torvalds@linux-foundation.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Thomas Gleixner <tglx@linutronix.de> Cc: Tomoki Sekiyama <tomoki.sekiyama.qu@hitachi.com> Link: http://lkml.kernel.org/r/20160811074430.GA18163@aaronlu.sh.intel.com Signed-off-by: Ingo Molnar <mingo@kernel.org> (cherry picked from commit 82ba4faca1bffad429f15c90c980ffd010366c25) Change-Id: Ie1344df19f91ef439f927b2be63fec8f4e35ac72 Reviewed-on: https://chromium-review.googlesource.com/742014 Commit-Ready: Jorge Lucangeli Obes <jorgelo@chromium.org> Tested-by: Jorge Lucangeli Obes <jorgelo@chromium.org> Reviewed-by: Guenter Roeck <groeck@chromium.org> [modify] https://crrev.com/14eed2a7509e56d48a3ac7b6b33cf2375840bc98/arch/x86/kernel/irq.c [modify] https://crrev.com/14eed2a7509e56d48a3ac7b6b33cf2375840bc98/arch/x86/include/asm/hardirq.h
,
Oct 27 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromiumos/third_party/kernel/+/84f6b8c4e6a4864c7f7979e18ed9c58780d06251 commit 84f6b8c4e6a4864c7f7979e18ed9c58780d06251 Author: Andy Lutomirski <luto@kernel.org> Date: Fri Oct 27 23:12:46 2017 UPSTREAM: x86/vm86/32: Switch to flush_tlb_mm_range() in mark_screen_rdonly() mark_screen_rdonly() is the last remaining caller of flush_tlb(). flush_tlb_mm_range() is potentially faster and isn't obsolete. Compile-tested only because I don't know whether software that uses this mechanism even exists. BUG=chromium:779077 TEST=Build, boot on caroline. Signed-off-by: Andy Lutomirski <luto@kernel.org> Cc: Andrew Morton <akpm@linux-foundation.org> Cc: Borislav Petkov <bp@alien8.de> Cc: Brian Gerst <brgerst@gmail.com> Cc: Dave Hansen <dave.hansen@intel.com> Cc: Denys Vlasenko <dvlasenk@redhat.com> Cc: H. Peter Anvin <hpa@zytor.com> Cc: Josh Poimboeuf <jpoimboe@redhat.com> Cc: Linus Torvalds <torvalds@linux-foundation.org> Cc: Michal Hocko <mhocko@suse.com> Cc: Nadav Amit <namit@vmware.com> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Rik van Riel <riel@redhat.com> Cc: Sasha Levin <sasha.levin@oracle.com> Cc: Thomas Gleixner <tglx@linutronix.de> Link: http://lkml.kernel.org/r/791a644076fc3577ba7f7b7cafd643cc089baa7d.1492844372.git.luto@kernel.org Signed-off-by: Ingo Molnar <mingo@kernel.org> (cherry picked from commit 9ccee2373f0658f234727700e619df097ba57023) Change-Id: I4d954a69c46567e6b36b2f239a34687121f11d4a Reviewed-on: https://chromium-review.googlesource.com/742015 Commit-Ready: Jorge Lucangeli Obes <jorgelo@chromium.org> Tested-by: Jorge Lucangeli Obes <jorgelo@chromium.org> Reviewed-by: Guenter Roeck <groeck@chromium.org> [modify] https://crrev.com/84f6b8c4e6a4864c7f7979e18ed9c58780d06251/arch/x86/kernel/vm86_32.c
,
Oct 27 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromiumos/third_party/kernel/+/eec0e9d5121bcdd9955da2e399b8afd75ce807cb commit eec0e9d5121bcdd9955da2e399b8afd75ce807cb Author: Andy Lutomirski <luto@kernel.org> Date: Fri Oct 27 23:12:47 2017 UPSTREAM: x86/mm: Remove flush_tlb() and flush_tlb_current_task() I was trying to figure out what how flush_tlb_current_task() would possibly work correctly if current->mm != current->active_mm, but I realized I could spare myself the effort: it has no callers except the unused flush_tlb() macro. BUG=chromium:779077 TEST=Build, boot on caroline. Signed-off-by: Andy Lutomirski <luto@kernel.org> Cc: Andrew Morton <akpm@linux-foundation.org> Cc: Borislav Petkov <bp@alien8.de> Cc: Brian Gerst <brgerst@gmail.com> Cc: Dave Hansen <dave.hansen@intel.com> Cc: Denys Vlasenko <dvlasenk@redhat.com> Cc: H. Peter Anvin <hpa@zytor.com> Cc: Josh Poimboeuf <jpoimboe@redhat.com> Cc: Linus Torvalds <torvalds@linux-foundation.org> Cc: Michal Hocko <mhocko@suse.com> Cc: Nadav Amit <namit@vmware.com> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Rik van Riel <riel@redhat.com> Cc: Thomas Gleixner <tglx@linutronix.de> Link: http://lkml.kernel.org/r/e52d64c11690f85e9f1d69d7b48cc2269cd2e94b.1492844372.git.luto@kernel.org Signed-off-by: Ingo Molnar <mingo@kernel.org> (cherry picked from commit 29961b59a51f8c6838a26a45e871a7ed6771809b) Change-Id: I96801f3da7943c0a3d949307355289cf1470a5d5 Reviewed-on: https://chromium-review.googlesource.com/742016 Commit-Ready: Jorge Lucangeli Obes <jorgelo@chromium.org> Tested-by: Jorge Lucangeli Obes <jorgelo@chromium.org> Reviewed-by: Guenter Roeck <groeck@chromium.org> [modify] https://crrev.com/eec0e9d5121bcdd9955da2e399b8afd75ce807cb/arch/x86/include/asm/tlbflush.h [modify] https://crrev.com/eec0e9d5121bcdd9955da2e399b8afd75ce807cb/arch/x86/mm/tlb.c
,
Oct 27 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromiumos/third_party/kernel/+/a753240ec6f826126b444c1b689419899faf4072 commit a753240ec6f826126b444c1b689419899faf4072 Author: Andy Lutomirski <luto@kernel.org> Date: Fri Oct 27 23:12:48 2017 UPSTREAM: x86/mm: Make flush_tlb_mm_range() more predictable I'm about to rewrite the function almost completely, but first I want to get a functional change out of the way. Currently, if flush_tlb_mm_range() does not flush the local TLB at all, it will never do individual page flushes on remote CPUs. This seems to be an accident, and preserving it will be awkward. Let's change it first so that any regressions in the rewrite will be easier to bisect and so that the rewrite can attempt to change no visible behavior at all. The fix is simple: we can simply avoid short-circuiting the calculation of base_pages_to_flush. As a side effect, this also eliminates a potential corner case: if tlb_single_page_flush_ceiling == TLB_FLUSH_ALL, flush_tlb_mm_range() could have ended up flushing the entire address space one page at a time. BUG=chromium:779077 TEST=Build, boot on caroline. Signed-off-by: Andy Lutomirski <luto@kernel.org> Acked-by: Dave Hansen <dave.hansen@intel.com> Cc: Andrew Morton <akpm@linux-foundation.org> Cc: Borislav Petkov <bp@alien8.de> Cc: Brian Gerst <brgerst@gmail.com> Cc: Denys Vlasenko <dvlasenk@redhat.com> Cc: H. Peter Anvin <hpa@zytor.com> Cc: Josh Poimboeuf <jpoimboe@redhat.com> Cc: Linus Torvalds <torvalds@linux-foundation.org> Cc: Michal Hocko <mhocko@suse.com> Cc: Nadav Amit <namit@vmware.com> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Rik van Riel <riel@redhat.com> Cc: Thomas Gleixner <tglx@linutronix.de> Link: http://lkml.kernel.org/r/4b29b771d9975aad7154c314534fec235618175a.1492844372.git.luto@kernel.org Signed-off-by: Ingo Molnar <mingo@kernel.org> (cherry picked from commit ce27374fabf553153c3f53efcaa9bfab9216bd8c) Change-Id: Ib0c14085117e89bc3b3465c095ef0a86fe0641ce Reviewed-on: https://chromium-review.googlesource.com/742017 Commit-Ready: Jorge Lucangeli Obes <jorgelo@chromium.org> Tested-by: Jorge Lucangeli Obes <jorgelo@chromium.org> Reviewed-by: Guenter Roeck <groeck@chromium.org> [modify] https://crrev.com/a753240ec6f826126b444c1b689419899faf4072/arch/x86/mm/tlb.c
,
Oct 27 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromiumos/third_party/kernel/+/5143c8ae62d5bb8f8e5447ad6d8e06d1792722f2 commit 5143c8ae62d5bb8f8e5447ad6d8e06d1792722f2 Author: Andy Lutomirski <luto@kernel.org> Date: Fri Oct 27 23:12:50 2017 UPSTREAM: x86/mm: Fix flush_tlb_page() on Xen flush_tlb_page() passes a bogus range to flush_tlb_others() and expects the latter to fix it up. native_flush_tlb_others() has the fixup but Xen's version doesn't. Move the fixup to flush_tlb_others(). AFAICS the only real effect is that, without this fix, Xen would flush everything instead of just the one page on remote vCPUs in when flush_tlb_page() was called. BUG=chromium:779077 TEST=Build, boot on caroline. Signed-off-by: Andy Lutomirski <luto@kernel.org> Reviewed-by: Boris Ostrovsky <boris.ostrovsky@oracle.com> Cc: Andrew Morton <akpm@linux-foundation.org> Cc: Borislav Petkov <bp@alien8.de> Cc: Brian Gerst <brgerst@gmail.com> Cc: Dave Hansen <dave.hansen@intel.com> Cc: Denys Vlasenko <dvlasenk@redhat.com> Cc: H. Peter Anvin <hpa@zytor.com> Cc: Josh Poimboeuf <jpoimboe@redhat.com> Cc: Juergen Gross <jgross@suse.com> Cc: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com> Cc: Linus Torvalds <torvalds@linux-foundation.org> Cc: Michal Hocko <mhocko@suse.com> Cc: Nadav Amit <namit@vmware.com> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Rik van Riel <riel@redhat.com> Cc: Thomas Gleixner <tglx@linutronix.de> Fixes: e7b52ffd45a6 ("x86/flush_tlb: try flush_tlb_single one by one in flush_tlb_range") Link: http://lkml.kernel.org/r/10ed0e4dfea64daef10b87fb85df1746999b4dba.1492844372.git.luto@kernel.org Signed-off-by: Ingo Molnar <mingo@kernel.org> (cherry picked from commit dbd68d8e84c606673ebbcf15862f8c155fa92326) Change-Id: Ie88fe5eebb01cd6ac7af138b5f2af0c47307d9df Reviewed-on: https://chromium-review.googlesource.com/742018 Commit-Ready: Jorge Lucangeli Obes <jorgelo@chromium.org> Tested-by: Jorge Lucangeli Obes <jorgelo@chromium.org> Reviewed-by: Guenter Roeck <groeck@chromium.org> [modify] https://crrev.com/5143c8ae62d5bb8f8e5447ad6d8e06d1792722f2/arch/x86/mm/tlb.c
,
Oct 27 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromiumos/third_party/kernel/+/4a4432bb04ffd3951958bd7ab258b37d147ab2ee commit 4a4432bb04ffd3951958bd7ab258b37d147ab2ee Author: Andy Lutomirski <luto@kernel.org> Date: Fri Oct 27 23:12:51 2017 UPSTREAM: x86/mm: Reimplement flush_tlb_page() using flush_tlb_mm_range() flush_tlb_page() was very similar to flush_tlb_mm_range() except that it had a couple of issues: - It was missing an smp_mb() in the case where current->active_mm != mm. (This is a longstanding bug reported by Nadav Amit) - It was missing tracepoints and vm counter updates. The only reason that I can see for keeping it at as a separate function is that it could avoid a few branches that flush_tlb_mm_range() needs to decide to flush just one page. This hardly seems worthwhile. If we decide we want to get rid of those branches again, a better way would be to introduce an __flush_tlb_mm_range() helper and make both flush_tlb_page() and flush_tlb_mm_range() use it. BUG=chromium:779077 TEST=Build, boot on caroline. Signed-off-by: Andy Lutomirski <luto@kernel.org> Acked-by: Kees Cook <keescook@chromium.org> Cc: Andrew Morton <akpm@linux-foundation.org> Cc: Borislav Petkov <bpetkov@suse.de> Cc: Dave Hansen <dave.hansen@intel.com> Cc: Linus Torvalds <torvalds@linux-foundation.org> Cc: Mel Gorman <mgorman@suse.de> Cc: Michal Hocko <mhocko@suse.com> Cc: Nadav Amit <nadav.amit@gmail.com> Cc: Nadav Amit <namit@vmware.com> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Rik van Riel <riel@redhat.com> Cc: Thomas Gleixner <tglx@linutronix.de> Cc: linux-mm@kvack.org Link: http://lkml.kernel.org/r/3cc3847cf888d8907577569b8bac3f01992ef8f9.1495492063.git.luto@kernel.org Signed-off-by: Ingo Molnar <mingo@kernel.org> (cherry picked from commit ca6c99c0794875c6d1db6e22f246699691ab7e6b) Change-Id: If7588124f68be57bf34c3292f457ff7a92bbd581 Reviewed-on: https://chromium-review.googlesource.com/742019 Commit-Ready: Jorge Lucangeli Obes <jorgelo@chromium.org> Tested-by: Jorge Lucangeli Obes <jorgelo@chromium.org> Reviewed-by: Guenter Roeck <groeck@chromium.org> [modify] https://crrev.com/4a4432bb04ffd3951958bd7ab258b37d147ab2ee/arch/x86/include/asm/tlbflush.h [modify] https://crrev.com/4a4432bb04ffd3951958bd7ab258b37d147ab2ee/arch/x86/mm/tlb.c
,
Oct 27 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromiumos/third_party/kernel/+/9a24c902abfa14cda403fb9a3d58e9d3a29d1c4c commit 9a24c902abfa14cda403fb9a3d58e9d3a29d1c4c Author: Andy Lutomirski <luto@kernel.org> Date: Fri Oct 27 23:12:52 2017 BACKPORT: x86/mm: Remove the UP asm/tlbflush.h code, always use the (formerly) SMP code The UP asm/tlbflush.h generates somewhat nicer code than the SMP version. Aside from that, it's fallen quite a bit behind the SMP code: - flush_tlb_mm_range() didn't flush individual pages if the range was small. - The lazy TLB code was much weaker. This usually wouldn't matter, but, if a kernel thread flushed its lazy "active_mm" more than once (due to reclaim or similar), it wouldn't be unlazied and would instead pointlessly flush repeatedly. - Tracepoints were missing. Aside from that, simply having the UP code around was a maintanence burden, since it means that any change to the TLB flush code had to make sure not to break it. Simplify everything by deleting the UP code. BUG=chromium:779077 TEST=Build, boot on caroline. Signed-off-by: Andy Lutomirski <luto@kernel.org> Cc: Andrew Morton <akpm@linux-foundation.org> Cc: Arjan van de Ven <arjan@linux.intel.com> Cc: Borislav Petkov <bpetkov@suse.de> Cc: Dave Hansen <dave.hansen@intel.com> Cc: Linus Torvalds <torvalds@linux-foundation.org> Cc: Mel Gorman <mgorman@suse.de> Cc: Michal Hocko <mhocko@suse.com> Cc: Nadav Amit <nadav.amit@gmail.com> Cc: Nadav Amit <namit@vmware.com> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Rik van Riel <riel@redhat.com> Cc: Thomas Gleixner <tglx@linutronix.de> Cc: linux-mm@kvack.org Signed-off-by: Ingo Molnar <mingo@kernel.org> (cherry picked from commit ce4a4e565f5264909a18c733b864c3f74467f69e) Conflicts: arch/x86/Kconfig arch/x86/include/asm/tlbbatch.h arch/x86/include/asm/tlbflush.h arch/x86/mm/tlb.c Change-Id: Ic698d21af4235540c79ca5c68d5e6d93d3242a8b Reviewed-on: https://chromium-review.googlesource.com/742020 Commit-Ready: Jorge Lucangeli Obes <jorgelo@chromium.org> Tested-by: Jorge Lucangeli Obes <jorgelo@chromium.org> Reviewed-by: Guenter Roeck <groeck@chromium.org> [modify] https://crrev.com/9a24c902abfa14cda403fb9a3d58e9d3a29d1c4c/arch/x86/include/asm/mmu_context.h [modify] https://crrev.com/9a24c902abfa14cda403fb9a3d58e9d3a29d1c4c/arch/x86/include/asm/hardirq.h [modify] https://crrev.com/9a24c902abfa14cda403fb9a3d58e9d3a29d1c4c/arch/x86/mm/init.c [modify] https://crrev.com/9a24c902abfa14cda403fb9a3d58e9d3a29d1c4c/arch/x86/include/asm/tlbflush.h [modify] https://crrev.com/9a24c902abfa14cda403fb9a3d58e9d3a29d1c4c/arch/x86/mm/tlb.c [modify] https://crrev.com/9a24c902abfa14cda403fb9a3d58e9d3a29d1c4c/arch/x86/include/asm/mmu.h
,
Oct 27 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromiumos/third_party/kernel/+/48b1d7645a26cbfa586a747030b872df67677850 commit 48b1d7645a26cbfa586a747030b872df67677850 Author: Andy Lutomirski <luto@kernel.org> Date: Fri Oct 27 23:12:53 2017 UPSTREAM: x86/mm: Disable PCID on 32-bit kernels 32-bit kernels on new hardware will see PCID in CPUID, but PCID can only be used in 64-bit mode. Rather than making all PCID code conditional, just disable the feature on 32-bit builds. BUG=chromium:779077 TEST=Build, boot on caroline. Signed-off-by: Andy Lutomirski <luto@kernel.org> Reviewed-by: Nadav Amit <nadav.amit@gmail.com> Reviewed-by: Borislav Petkov <bp@suse.de> Reviewed-by: Thomas Gleixner <tglx@linutronix.de> Cc: Andrew Morton <akpm@linux-foundation.org> Cc: Arjan van de Ven <arjan@linux.intel.com> Cc: Borislav Petkov <bp@alien8.de> Cc: Dave Hansen <dave.hansen@intel.com> Cc: Linus Torvalds <torvalds@linux-foundation.org> Cc: Mel Gorman <mgorman@suse.de> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Rik van Riel <riel@redhat.com> Cc: linux-mm@kvack.org Link: http://lkml.kernel.org/r/2e391769192a4d31b808410c383c6bf0734bc6ea.1498751203.git.luto@kernel.org Signed-off-by: Ingo Molnar <mingo@kernel.org> (cherry picked from commit cba4671af7550e008f7a7835f06df0763825bf3e) Change-Id: Iad0f97cfa22b4c9924cfea5e7678a36e8e81a65e Reviewed-on: https://chromium-review.googlesource.com/742021 Commit-Ready: Jorge Lucangeli Obes <jorgelo@chromium.org> Tested-by: Jorge Lucangeli Obes <jorgelo@chromium.org> Reviewed-by: Guenter Roeck <groeck@chromium.org> [modify] https://crrev.com/48b1d7645a26cbfa586a747030b872df67677850/arch/x86/include/asm/disabled-features.h [modify] https://crrev.com/48b1d7645a26cbfa586a747030b872df67677850/arch/x86/kernel/cpu/bugs.c
,
Oct 27 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromiumos/third_party/kernel/+/c4ddcd3c20fdb309854cdfa172c09e6e45170cdc commit c4ddcd3c20fdb309854cdfa172c09e6e45170cdc Author: Andy Lutomirski <luto@kernel.org> Date: Fri Oct 27 23:12:54 2017 BACKPORT: x86/mm: Add the 'nopcid' boot option to turn off PCID The parameter is only present on x86_64 systems to save a few bytes, as PCID is always disabled on x86_32. BUG=chromium:779077 TEST=Build, boot on caroline. Signed-off-by: Andy Lutomirski <luto@kernel.org> Reviewed-by: Nadav Amit <nadav.amit@gmail.com> Reviewed-by: Borislav Petkov <bp@suse.de> Reviewed-by: Thomas Gleixner <tglx@linutronix.de> Cc: Andrew Morton <akpm@linux-foundation.org> Cc: Arjan van de Ven <arjan@linux.intel.com> Cc: Borislav Petkov <bp@alien8.de> Cc: Dave Hansen <dave.hansen@intel.com> Cc: Linus Torvalds <torvalds@linux-foundation.org> Cc: Mel Gorman <mgorman@suse.de> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Rik van Riel <riel@redhat.com> Cc: linux-mm@kvack.org Link: http://lkml.kernel.org/r/8bbb2e65bcd249a5f18bfb8128b4689f08ac2b60.1498751203.git.luto@kernel.org Signed-off-by: Ingo Molnar <mingo@kernel.org> (cherry picked from commit 0790c9aad84901ca1bdc14746175549c8b5da215) Conflicts: Documentation/admin-guide/kernel-parameters.txt (not in this tree) Documentation/kernel-parameters.txt (patched instead of that) Change-Id: I8e87aa6680b2586729ef1ed6f83626b90330dfed Reviewed-on: https://chromium-review.googlesource.com/742022 Commit-Ready: Jorge Lucangeli Obes <jorgelo@chromium.org> Tested-by: Jorge Lucangeli Obes <jorgelo@chromium.org> Reviewed-by: Guenter Roeck <groeck@chromium.org> [modify] https://crrev.com/c4ddcd3c20fdb309854cdfa172c09e6e45170cdc/Documentation/kernel-parameters.txt [modify] https://crrev.com/c4ddcd3c20fdb309854cdfa172c09e6e45170cdc/arch/x86/kernel/cpu/common.c
,
Oct 27 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromiumos/third_party/kernel/+/aeed71350de7d2cd14ab491028ccb4b0d085ddd4 commit aeed71350de7d2cd14ab491028ccb4b0d085ddd4 Author: Andy Lutomirski <luto@kernel.org> Date: Fri Oct 27 23:12:55 2017 BACKPORT: x86/mm: Enable CR4.PCIDE on supported systems We can use PCID if the CPU has PCID and PGE and we're not on Xen. By itself, this has no effect. A followup patch will start using PCID. BUG=chromium:779077 TEST=Build, boot on caroline. Signed-off-by: Andy Lutomirski <luto@kernel.org> Reviewed-by: Nadav Amit <nadav.amit@gmail.com> Reviewed-by: Boris Ostrovsky <boris.ostrovsky@oracle.com> Reviewed-by: Thomas Gleixner <tglx@linutronix.de> Cc: Andrew Morton <akpm@linux-foundation.org> Cc: Arjan van de Ven <arjan@linux.intel.com> Cc: Borislav Petkov <bp@alien8.de> Cc: Dave Hansen <dave.hansen@intel.com> Cc: Juergen Gross <jgross@suse.com> Cc: Linus Torvalds <torvalds@linux-foundation.org> Cc: Mel Gorman <mgorman@suse.de> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Rik van Riel <riel@redhat.com> Cc: linux-mm@kvack.org Link: http://lkml.kernel.org/r/6327ecd907b32f79d5aa0d466f04503bbec5df88.1498751203.git.luto@kernel.org Signed-off-by: Ingo Molnar <mingo@kernel.org> (cherry picked from commit 660da7c9228f685b2ebe664f9fd69aaddcc420b5) Conflicts: arch/x86/xen/enlighten_pv.c (not in this tree) arch/x86/xen/enlighten.c (patched instead of that) Change-Id: I9c3316f75c91c8ab831ce3fe55a283e74189f3c2 Reviewed-on: https://chromium-review.googlesource.com/742023 Commit-Ready: Jorge Lucangeli Obes <jorgelo@chromium.org> Tested-by: Jorge Lucangeli Obes <jorgelo@chromium.org> Reviewed-by: Guenter Roeck <groeck@chromium.org> [modify] https://crrev.com/aeed71350de7d2cd14ab491028ccb4b0d085ddd4/arch/x86/include/asm/tlbflush.h [modify] https://crrev.com/aeed71350de7d2cd14ab491028ccb4b0d085ddd4/arch/x86/xen/enlighten.c [modify] https://crrev.com/aeed71350de7d2cd14ab491028ccb4b0d085ddd4/arch/x86/kernel/cpu/common.c
,
Oct 27 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromiumos/third_party/kernel/+/1d00ad67ff6c91329e4d16f6d943d877429fcf9e commit 1d00ad67ff6c91329e4d16f6d943d877429fcf9e Author: Andy Lutomirski <luto@kernel.org> Date: Fri Oct 27 23:13:04 2017 UPSTREAM: sched/core: Add switch_mm_irqs_off() and use it in the scheduler By default, this is the same thing as switch_mm(). x86 will override it as an optimization. BUG=chromium:779077 TEST=Build, boot on soraka. Signed-off-by: Andy Lutomirski <luto@kernel.org> Reviewed-by: Borislav Petkov <bp@suse.de> Cc: Borislav Petkov <bp@alien8.de> Cc: Linus Torvalds <torvalds@linux-foundation.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Thomas Gleixner <tglx@linutronix.de> Link: http://lkml.kernel.org/r/df401df47bdd6be3e389c6f1e3f5310d70e81b2c.1461688545.git.luto@kernel.org Signed-off-by: Ingo Molnar <mingo@kernel.org> (cherry picked from commit f98db6013c557c216da5038d9c52045be55cd039) Change-Id: Ice480e7d97c49fca442dbcb2ff32861055ef8ddb Reviewed-on: https://chromium-review.googlesource.com/741902 Commit-Ready: Jorge Lucangeli Obes <jorgelo@chromium.org> Tested-by: Jorge Lucangeli Obes <jorgelo@chromium.org> Reviewed-by: Guenter Roeck <groeck@chromium.org> [modify] https://crrev.com/1d00ad67ff6c91329e4d16f6d943d877429fcf9e/include/linux/mmu_context.h [modify] https://crrev.com/1d00ad67ff6c91329e4d16f6d943d877429fcf9e/kernel/sched/core.c
,
Oct 27 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromiumos/third_party/kernel/+/2ee9dcbaa02b3c381040711d0348b68ff8025827 commit 2ee9dcbaa02b3c381040711d0348b68ff8025827 Author: Andy Lutomirski <luto@kernel.org> Date: Fri Oct 27 23:13:05 2017 UPSTREAM: x86/mm: Build arch/x86/mm/tlb.c even on !SMP Currently all of the functions that live in tlb.c are inlined on !SMP builds. One can debate whether this is a good idea (in many respects the code in tlb.c is better than the inlined UP code). Regardless, I want to add code that needs to be built on UP and SMP kernels and relates to tlb flushing, so arrange for tlb.c to be compiled unconditionally. BUG=chromium:779077 TEST=Build, boot on soraka. Signed-off-by: Andy Lutomirski <luto@kernel.org> Reviewed-by: Borislav Petkov <bp@suse.de> Cc: Borislav Petkov <bp@alien8.de> Cc: Linus Torvalds <torvalds@linux-foundation.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Thomas Gleixner <tglx@linutronix.de> Link: http://lkml.kernel.org/r/f0d778f0d828fc46e5d1946bca80f0aaf9abf032.1461688545.git.luto@kernel.org Signed-off-by: Ingo Molnar <mingo@kernel.org> (cherry picked from commit e1074888c326038340a1ada9129d679e661f2ea6) Change-Id: If7e0e98783e4c7ced3224e0ef65c9f162468947c Reviewed-on: https://chromium-review.googlesource.com/741903 Commit-Ready: Jorge Lucangeli Obes <jorgelo@chromium.org> Tested-by: Jorge Lucangeli Obes <jorgelo@chromium.org> Reviewed-by: Guenter Roeck <groeck@chromium.org> [modify] https://crrev.com/2ee9dcbaa02b3c381040711d0348b68ff8025827/arch/x86/mm/tlb.c [modify] https://crrev.com/2ee9dcbaa02b3c381040711d0348b68ff8025827/arch/x86/mm/Makefile
,
Nov 9 2017
,
Nov 9 2017
The bug is marked as P3 or Feature. It should not be merged as M63 is in beta. Please contact the approriate milestone owner if you have questions. Owners: cmasso@(Android), cmasso@(iOS), gkihumba@(ChromeOS), govind@(Desktop) For more details visit https://www.chromium.org/issue-tracking/autotriage - Your friendly Sheriffbot
,
Nov 9 2017
Well sheriffbot is not helping.
,
Nov 9 2017
Surprise.
,
Nov 9 2017
This bug requires manual review: M63 has already been promoted to the beta branch, so this requires manual review Please contact the milestone owner if you have questions. Owners: cmasso@(Android), cmasso@(iOS), gkihumba@(ChromeOS), govind@(Desktop) For more details visit https://www.chromium.org/issue-tracking/autotriage - Your friendly Sheriffbot
,
Nov 9 2017
Merge approved for first set of patches.
,
Nov 10 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromiumos/third_party/kernel/+/97ef7c81d086fd480a89a080a61f05195b05da83 commit 97ef7c81d086fd480a89a080a61f05195b05da83 Author: Andy Lutomirski <luto@kernel.org> Date: Fri Nov 10 21:12:50 2017 UPSTREAM: x86/mm: Add INVPCID helpers This adds helpers for each of the four currently-specified INVPCID modes. BUG=chromium:779077 TEST=Build, boot on soraka. Signed-off-by: Andy Lutomirski <luto@kernel.org> Reviewed-by: Borislav Petkov <bp@suse.de> Cc: Andrew Morton <akpm@linux-foundation.org> Cc: Andrey Ryabinin <aryabinin@virtuozzo.com> Cc: Andy Lutomirski <luto@amacapital.net> Cc: Borislav Petkov <bp@alien8.de> Cc: Brian Gerst <brgerst@gmail.com> Cc: Dave Hansen <dave.hansen@linux.intel.com> Cc: Denys Vlasenko <dvlasenk@redhat.com> Cc: H. Peter Anvin <hpa@zytor.com> Cc: Linus Torvalds <torvalds@linux-foundation.org> Cc: Luis R. Rodriguez <mcgrof@suse.com> Cc: Oleg Nesterov <oleg@redhat.com> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Thomas Gleixner <tglx@linutronix.de> Cc: Toshi Kani <toshi.kani@hp.com> Cc: linux-mm@kvack.org Link: http://lkml.kernel.org/r/8a62b23ad686888cee01da134c91409e22064db9.1454096309.git.luto@kernel.org Signed-off-by: Ingo Molnar <mingo@kernel.org> (cherry picked from commit 060a402a1ddb551455ee410de2eadd3349f2801b) Change-Id: I7a345e3cb9cd8264e25a7a438f8205bbe599c8a1 Reviewed-on: https://chromium-review.googlesource.com/741336 Commit-Ready: Jorge Lucangeli Obes <jorgelo@chromium.org> Tested-by: Jorge Lucangeli Obes <jorgelo@chromium.org> Reviewed-by: Guenter Roeck <groeck@chromium.org> (cherry picked from commit c8ba9445830fb247f20e1bc653ee15c567eb5980) Signed-off-by: Aditya Kali <adityakali@google.com> Reviewed-on: https://chromium-review.googlesource.com/762045 Reviewed-by: Jorge Lucangeli Obes <jorgelo@chromium.org> Commit-Queue: Jorge Lucangeli Obes <jorgelo@chromium.org> [modify] https://crrev.com/97ef7c81d086fd480a89a080a61f05195b05da83/arch/x86/include/asm/tlbflush.h
,
Nov 10 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromiumos/third_party/kernel/+/e9ab18970bdb90149fb889bdf7a041c67a5057d1 commit e9ab18970bdb90149fb889bdf7a041c67a5057d1 Author: Borislav Petkov <bp@suse.de> Date: Fri Nov 10 21:13:05 2017 UPSTREAM: x86/mm: Fix INVPCID asm constraint So we want to specify the dependency on both @pcid and @addr so that the compiler doesn't reorder accesses to them *before* the TLB flush. But for that to work, we need to express this properly in the inline asm and deref the whole desc array, not the pointer to it. See clwb() for an example. This fixes the build error on 32-bit: arch/x86/include/asm/tlbflush.h: In function __invpcid: arch/x86/include/asm/tlbflush.h:26:18: error: memory input 0 is not directly addressable which gcc4.7 caught but 5.x didn't. Which is strange. :-\ BUG=chromium:779077 TEST=Build, boot on soraka. Signed-off-by: Borislav Petkov <bp@suse.de> Cc: Andrew Morton <akpm@linux-foundation.org> Cc: Andrey Ryabinin <aryabinin@virtuozzo.com> Cc: Andy Lutomirski <luto@amacapital.net> Cc: Borislav Petkov <bp@alien8.de> Cc: Brian Gerst <brgerst@gmail.com> Cc: Dave Hansen <dave.hansen@linux.intel.com> Cc: Denys Vlasenko <dvlasenk@redhat.com> Cc: H. Peter Anvin <hpa@zytor.com> Cc: Linus Torvalds <torvalds@linux-foundation.org> Cc: Luis R. Rodriguez <mcgrof@suse.com> Cc: Michael Matz <matz@suse.de> Cc: Oleg Nesterov <oleg@redhat.com> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Thomas Gleixner <tglx@linutronix.de> Cc: Toshi Kani <toshi.kani@hp.com> Cc: linux-mm@kvack.org Signed-off-by: Ingo Molnar <mingo@kernel.org> (cherry picked from commit e2c7698cd61f11d4077fdb28148b2d31b82ac848) Change-Id: I0f63f22403709f3aacaa9f3406ba67260c31acef Reviewed-on: https://chromium-review.googlesource.com/741337 Commit-Ready: Jorge Lucangeli Obes <jorgelo@chromium.org> Tested-by: Jorge Lucangeli Obes <jorgelo@chromium.org> Reviewed-by: Guenter Roeck <groeck@chromium.org> (cherry picked from commit 46beb28b260f32301d03edd8d94f71af5de56dc1) Signed-off-by: Aditya Kali <adityakali@google.com> Reviewed-on: https://chromium-review.googlesource.com/762046 Reviewed-by: Jorge Lucangeli Obes <jorgelo@chromium.org> Commit-Queue: Jorge Lucangeli Obes <jorgelo@chromium.org> [modify] https://crrev.com/e9ab18970bdb90149fb889bdf7a041c67a5057d1/arch/x86/include/asm/tlbflush.h
,
Nov 10 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromiumos/third_party/kernel/+/7a435891eba8058016b98614df6680e805d2ff97 commit 7a435891eba8058016b98614df6680e805d2ff97 Author: Andy Lutomirski <luto@kernel.org> Date: Fri Nov 10 21:13:10 2017 UPSTREAM: x86/mm: Add a 'noinvpcid' boot option to turn off INVPCID This adds a chicken bit to turn off INVPCID in case something goes wrong. It's an early_param() because we do TLB flushes before we parse __setup() parameters. BUG=chromium:779077 TEST=Build, boot on soraka. Signed-off-by: Andy Lutomirski <luto@kernel.org> Reviewed-by: Borislav Petkov <bp@suse.de> Cc: Andrew Morton <akpm@linux-foundation.org> Cc: Andrey Ryabinin <aryabinin@virtuozzo.com> Cc: Andy Lutomirski <luto@amacapital.net> Cc: Borislav Petkov <bp@alien8.de> Cc: Brian Gerst <brgerst@gmail.com> Cc: Dave Hansen <dave.hansen@linux.intel.com> Cc: Denys Vlasenko <dvlasenk@redhat.com> Cc: H. Peter Anvin <hpa@zytor.com> Cc: Linus Torvalds <torvalds@linux-foundation.org> Cc: Luis R. Rodriguez <mcgrof@suse.com> Cc: Oleg Nesterov <oleg@redhat.com> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Thomas Gleixner <tglx@linutronix.de> Cc: Toshi Kani <toshi.kani@hp.com> Cc: linux-mm@kvack.org Link: http://lkml.kernel.org/r/f586317ed1bc2b87aee652267e515b90051af385.1454096309.git.luto@kernel.org Signed-off-by: Ingo Molnar <mingo@kernel.org> (cherry picked from commit d12a72b844a49d4162f24cefdab30bed3f86730e) Change-Id: Iacab182f0c700c43a2f12cf14a0090751665c626 Reviewed-on: https://chromium-review.googlesource.com/741338 Commit-Ready: Jorge Lucangeli Obes <jorgelo@chromium.org> Tested-by: Jorge Lucangeli Obes <jorgelo@chromium.org> Reviewed-by: Guenter Roeck <groeck@chromium.org> (cherry picked from commit 890bf6279732cba34c9c31e0a547e5b0c2862a4a) Signed-off-by: Aditya Kali <adityakali@google.com> Reviewed-on: https://chromium-review.googlesource.com/762047 Reviewed-by: Jorge Lucangeli Obes <jorgelo@chromium.org> Commit-Queue: Jorge Lucangeli Obes <jorgelo@chromium.org> [modify] https://crrev.com/7a435891eba8058016b98614df6680e805d2ff97/Documentation/kernel-parameters.txt [modify] https://crrev.com/7a435891eba8058016b98614df6680e805d2ff97/arch/x86/kernel/cpu/common.c
,
Nov 10 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromiumos/third_party/kernel/+/e25def0cf31161fddf953d199d3e12b0454a1a8c commit e25def0cf31161fddf953d199d3e12b0454a1a8c Author: Andy Lutomirski <luto@kernel.org> Date: Fri Nov 10 21:13:23 2017 UPSTREAM: x86/mm: If INVPCID is available, use it to flush global mappings On my Skylake laptop, INVPCID function 2 (flush absolutely everything) takes about 376ns, whereas saving flags, twiddling CR4.PGE to flush global mappings, and restoring flags takes about 539ns. BUG=chromium:779077 TEST=Build, boot on soraka. Signed-off-by: Andy Lutomirski <luto@kernel.org> Reviewed-by: Borislav Petkov <bp@suse.de> Cc: Andrew Morton <akpm@linux-foundation.org> Cc: Andrey Ryabinin <aryabinin@virtuozzo.com> Cc: Andy Lutomirski <luto@amacapital.net> Cc: Borislav Petkov <bp@alien8.de> Cc: Brian Gerst <brgerst@gmail.com> Cc: Dave Hansen <dave.hansen@linux.intel.com> Cc: Denys Vlasenko <dvlasenk@redhat.com> Cc: H. Peter Anvin <hpa@zytor.com> Cc: Linus Torvalds <torvalds@linux-foundation.org> Cc: Luis R. Rodriguez <mcgrof@suse.com> Cc: Oleg Nesterov <oleg@redhat.com> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Thomas Gleixner <tglx@linutronix.de> Cc: Toshi Kani <toshi.kani@hp.com> Cc: linux-mm@kvack.org Link: http://lkml.kernel.org/r/ed0ef62581c0ea9c99b9bf6df726015e96d44743.1454096309.git.luto@kernel.org Signed-off-by: Ingo Molnar <mingo@kernel.org> (cherry picked from commit d8bced79af1db6734f66b42064cc773cada2ce99) Change-Id: I0ff9813f164963ae89520d4c995bd937b045863f Reviewed-on: https://chromium-review.googlesource.com/741339 Commit-Ready: Jorge Lucangeli Obes <jorgelo@chromium.org> Tested-by: Jorge Lucangeli Obes <jorgelo@chromium.org> Reviewed-by: Guenter Roeck <groeck@chromium.org> (cherry picked from commit 5a74c34a7b20d224874d9956c434b90563341f93) Signed-off-by: Aditya Kali <adityakali@google.com> Reviewed-on: https://chromium-review.googlesource.com/762048 Reviewed-by: Jorge Lucangeli Obes <jorgelo@chromium.org> Commit-Queue: Jorge Lucangeli Obes <jorgelo@chromium.org> [modify] https://crrev.com/e25def0cf31161fddf953d199d3e12b0454a1a8c/arch/x86/include/asm/tlbflush.h
,
Nov 10 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromiumos/third_party/kernel/+/d34417e973fb373f7ea6619761b4f30dc4ce5dab commit d34417e973fb373f7ea6619761b4f30dc4ce5dab Author: Nadav Amit <namit@vmware.com> Date: Fri Nov 10 21:13:28 2017 UPSTREAM: mm/rmap: batched invalidations should use existing api The recently introduced batched invalidations mechanism uses its own mechanism for shootdown. However, it does wrong accounting of interrupts (e.g., inc_irq_stat is called for local invalidations), trace-points (e.g., TLB_REMOTE_SHOOTDOWN for local invalidations) and may break some platforms as it bypasses the invalidation mechanisms of Xen and SGI UV. This patch reuses the existing TLB flushing mechnaisms instead. We use NULL as mm to indicate a global invalidation is required. BUG=chromium:779077 TEST=Build, boot on soraka. Fixes 72b252aed506b8 ("mm: send one IPI per CPU to TLB flush all entries after unmapping pages") Signed-off-by: Nadav Amit <namit@vmware.com> Cc: Mel Gorman <mgorman@suse.de> Cc: Rik van Riel <riel@redhat.com> Cc: Dave Hansen <dave.hansen@intel.com> Cc: Ingo Molnar <mingo@kernel.org> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org> (cherry picked from commit 858eaaa711700ce4595e039441e239e56d7b9514) Change-Id: Ia30c987d370400879c8afc80276cabe1a06e9c1e Reviewed-on: https://chromium-review.googlesource.com/741340 Commit-Ready: Jorge Lucangeli Obes <jorgelo@chromium.org> Tested-by: Jorge Lucangeli Obes <jorgelo@chromium.org> Reviewed-by: Guenter Roeck <groeck@chromium.org> (cherry picked from commit 86a09af748d06812d9759225384a1649733c2afc) Signed-off-by: Aditya Kali <adityakali@google.com> Reviewed-on: https://chromium-review.googlesource.com/762049 Reviewed-by: Jorge Lucangeli Obes <jorgelo@chromium.org> Commit-Queue: Jorge Lucangeli Obes <jorgelo@chromium.org> [modify] https://crrev.com/d34417e973fb373f7ea6619761b4f30dc4ce5dab/arch/x86/include/asm/tlbflush.h [modify] https://crrev.com/d34417e973fb373f7ea6619761b4f30dc4ce5dab/arch/x86/mm/tlb.c [modify] https://crrev.com/d34417e973fb373f7ea6619761b4f30dc4ce5dab/mm/rmap.c
,
Nov 10 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromiumos/third_party/kernel/+/a4c1194d82689ff52f4183d264b2e77eaee6b5e4 commit a4c1194d82689ff52f4183d264b2e77eaee6b5e4 Author: Ingo Molnar <mingo@kernel.org> Date: Fri Nov 10 21:13:32 2017 UPSTREAM: mm/mmu_context, sched/core: Fix mmu_context.h assumption Some architectures (such as Alpha) rely on include/linux/sched.h definitions in their mmu_context.h files. So include sched.h before mmu_context.h. BUG=chromium:779077 TEST=Build, boot on soraka. Cc: Andy Lutomirski <luto@kernel.org> Cc: Borislav Petkov <bp@alien8.de> Cc: Linus Torvalds <torvalds@linux-foundation.org> Cc: linux-kernel@vger.kernel.org Cc: Peter Zijlstra <peterz@infradead.org> Cc: Thomas Gleixner <tglx@linutronix.de> Signed-off-by: Ingo Molnar <mingo@kernel.org> (cherry picked from commit 8efd755ac2fe262d4c8d5c9bbe054bb67dae93da) Change-Id: I59ef1d786b1ec996cff98355325c39e47ce5ca69 Reviewed-on: https://chromium-review.googlesource.com/741901 Commit-Ready: Jorge Lucangeli Obes <jorgelo@chromium.org> Tested-by: Jorge Lucangeli Obes <jorgelo@chromium.org> Reviewed-by: Guenter Roeck <groeck@chromium.org> (cherry picked from commit 8246978a5ff7f100d403aa07f24e9cd69388d522) Signed-off-by: Aditya Kali <adityakali@google.com> Reviewed-on: https://chromium-review.googlesource.com/762050 Reviewed-by: Jorge Lucangeli Obes <jorgelo@chromium.org> Commit-Queue: Jorge Lucangeli Obes <jorgelo@chromium.org> [modify] https://crrev.com/a4c1194d82689ff52f4183d264b2e77eaee6b5e4/mm/mmu_context.c
,
Nov 10 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromiumos/third_party/kernel/+/559bfac467d7d72367a91eeed0909afe924a6c18 commit 559bfac467d7d72367a91eeed0909afe924a6c18 Author: Andy Lutomirski <luto@kernel.org> Date: Fri Nov 10 21:13:36 2017 UPSTREAM: sched/core: Add switch_mm_irqs_off() and use it in the scheduler By default, this is the same thing as switch_mm(). x86 will override it as an optimization. BUG=chromium:779077 TEST=Build, boot on soraka. Signed-off-by: Andy Lutomirski <luto@kernel.org> Reviewed-by: Borislav Petkov <bp@suse.de> Cc: Borislav Petkov <bp@alien8.de> Cc: Linus Torvalds <torvalds@linux-foundation.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Thomas Gleixner <tglx@linutronix.de> Link: http://lkml.kernel.org/r/df401df47bdd6be3e389c6f1e3f5310d70e81b2c.1461688545.git.luto@kernel.org Signed-off-by: Ingo Molnar <mingo@kernel.org> (cherry picked from commit f98db6013c557c216da5038d9c52045be55cd039) Change-Id: Ice480e7d97c49fca442dbcb2ff32861055ef8ddb Reviewed-on: https://chromium-review.googlesource.com/741902 Commit-Ready: Jorge Lucangeli Obes <jorgelo@chromium.org> Tested-by: Jorge Lucangeli Obes <jorgelo@chromium.org> Reviewed-by: Guenter Roeck <groeck@chromium.org> (cherry picked from commit 1d00ad67ff6c91329e4d16f6d943d877429fcf9e) Signed-off-by: Aditya Kali <adityakali@google.com> Reviewed-on: https://chromium-review.googlesource.com/762051 Reviewed-by: Jorge Lucangeli Obes <jorgelo@chromium.org> Commit-Queue: Jorge Lucangeli Obes <jorgelo@chromium.org> [modify] https://crrev.com/559bfac467d7d72367a91eeed0909afe924a6c18/include/linux/mmu_context.h [modify] https://crrev.com/559bfac467d7d72367a91eeed0909afe924a6c18/kernel/sched/core.c
,
Nov 10 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromiumos/third_party/kernel/+/68659541b4c68511f3d2d631495ec8cfc909f49d commit 68659541b4c68511f3d2d631495ec8cfc909f49d Author: Andy Lutomirski <luto@kernel.org> Date: Fri Nov 10 21:13:41 2017 UPSTREAM: x86/mm: Build arch/x86/mm/tlb.c even on !SMP Currently all of the functions that live in tlb.c are inlined on !SMP builds. One can debate whether this is a good idea (in many respects the code in tlb.c is better than the inlined UP code). Regardless, I want to add code that needs to be built on UP and SMP kernels and relates to tlb flushing, so arrange for tlb.c to be compiled unconditionally. BUG=chromium:779077 TEST=Build, boot on soraka. Signed-off-by: Andy Lutomirski <luto@kernel.org> Reviewed-by: Borislav Petkov <bp@suse.de> Cc: Borislav Petkov <bp@alien8.de> Cc: Linus Torvalds <torvalds@linux-foundation.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Thomas Gleixner <tglx@linutronix.de> Link: http://lkml.kernel.org/r/f0d778f0d828fc46e5d1946bca80f0aaf9abf032.1461688545.git.luto@kernel.org Signed-off-by: Ingo Molnar <mingo@kernel.org> (cherry picked from commit e1074888c326038340a1ada9129d679e661f2ea6) Change-Id: If7e0e98783e4c7ced3224e0ef65c9f162468947c Reviewed-on: https://chromium-review.googlesource.com/741903 Commit-Ready: Jorge Lucangeli Obes <jorgelo@chromium.org> Tested-by: Jorge Lucangeli Obes <jorgelo@chromium.org> Reviewed-by: Guenter Roeck <groeck@chromium.org> (cherry picked from commit 2ee9dcbaa02b3c381040711d0348b68ff8025827) Signed-off-by: Aditya Kali <adityakali@google.com> Reviewed-on: https://chromium-review.googlesource.com/762052 Reviewed-by: Jorge Lucangeli Obes <jorgelo@chromium.org> Commit-Queue: Jorge Lucangeli Obes <jorgelo@chromium.org> [modify] https://crrev.com/68659541b4c68511f3d2d631495ec8cfc909f49d/arch/x86/mm/tlb.c [modify] https://crrev.com/68659541b4c68511f3d2d631495ec8cfc909f49d/arch/x86/mm/Makefile
,
Nov 10 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromiumos/third_party/kernel/+/664634847832cf8e848413b9e3c5370429528d87 commit 664634847832cf8e848413b9e3c5370429528d87 Author: Andy Lutomirski <luto@kernel.org> Date: Fri Nov 10 21:13:45 2017 BACKPORT: x86/mm, sched/core: Uninline switch_mm() It's fairly large and it has quite a few callers. This may also help untangle some headers down the road. BUG=chromium:779077 TEST=Build, boot on soraka. Signed-off-by: Andy Lutomirski <luto@kernel.org> Reviewed-by: Borislav Petkov <bp@suse.de> Cc: Borislav Petkov <bp@alien8.de> Cc: Linus Torvalds <torvalds@linux-foundation.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Thomas Gleixner <tglx@linutronix.de> Link: http://lkml.kernel.org/r/54f3367803e7f80b2be62c8a21879aa74b1a5f57.1461688545.git.luto@kernel.org Signed-off-by: Ingo Molnar <mingo@kernel.org> (cherry picked from commit 69c0319aabba45bcf33178916a2f06967b4adede) Conflicts: arch/x86/include/asm/mmu_context.h Change-Id: Ib702286e2bedd74723708d0518a07d2cf5034c94 Reviewed-on: https://chromium-review.googlesource.com/741904 Commit-Ready: Jorge Lucangeli Obes <jorgelo@chromium.org> Tested-by: Jorge Lucangeli Obes <jorgelo@chromium.org> Reviewed-by: Guenter Roeck <groeck@chromium.org> (cherry picked from commit e311e1aca74be051a8ff7d6ca3c53a62137d2722) Signed-off-by: Aditya Kali <adityakali@google.com> Reviewed-on: https://chromium-review.googlesource.com/762053 Reviewed-by: Jorge Lucangeli Obes <jorgelo@chromium.org> Commit-Queue: Jorge Lucangeli Obes <jorgelo@chromium.org> [modify] https://crrev.com/664634847832cf8e848413b9e3c5370429528d87/arch/x86/include/asm/mmu_context.h [modify] https://crrev.com/664634847832cf8e848413b9e3c5370429528d87/arch/x86/mm/tlb.c
,
Nov 10 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromiumos/third_party/kernel/+/9bc9414fa1a392cba27798577bfc240ec03e1d94 commit 9bc9414fa1a392cba27798577bfc240ec03e1d94 Author: Andy Lutomirski <luto@kernel.org> Date: Fri Nov 10 21:13:50 2017 UPSTREAM: x86/mm, sched/core: Turn off IRQs in switch_mm() Potential races between switch_mm() and TLB-flush or LDT-flush IPIs could be very messy. AFAICT the code is currently okay, whether by accident or by careful design, but enabling PCID will make it considerably more complicated and will no longer be obviously safe. Fix it with a big hammer: run switch_mm() with IRQs off. To avoid a performance hit in the scheduler, we take advantage of our knowledge that the scheduler already has IRQs disabled when it calls switch_mm(). BUG=chromium:779077 TEST=Build, boot on soraka. Signed-off-by: Andy Lutomirski <luto@kernel.org> Reviewed-by: Borislav Petkov <bp@suse.de> Cc: Borislav Petkov <bp@alien8.de> Cc: Linus Torvalds <torvalds@linux-foundation.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Thomas Gleixner <tglx@linutronix.de> Link: http://lkml.kernel.org/r/f19baf759693c9dcae64bbff76189db77cb13398.1461688545.git.luto@kernel.org Signed-off-by: Ingo Molnar <mingo@kernel.org> (cherry picked from commit 078194f8e9fe3cf54c8fd8bded48a1db5bd8eb8a) Change-Id: Ie46238916f004d6a5e81fc03c0701cd187745ef7 Reviewed-on: https://chromium-review.googlesource.com/741905 Commit-Ready: Jorge Lucangeli Obes <jorgelo@chromium.org> Tested-by: Jorge Lucangeli Obes <jorgelo@chromium.org> Reviewed-by: Guenter Roeck <groeck@chromium.org> (cherry picked from commit e1c536b810740c75d7463f16f317e2b486f041af) Signed-off-by: Aditya Kali <adityakali@google.com> Reviewed-on: https://chromium-review.googlesource.com/762054 Reviewed-by: Jorge Lucangeli Obes <jorgelo@chromium.org> Commit-Queue: Jorge Lucangeli Obes <jorgelo@chromium.org> [modify] https://crrev.com/9bc9414fa1a392cba27798577bfc240ec03e1d94/arch/x86/include/asm/mmu_context.h [modify] https://crrev.com/9bc9414fa1a392cba27798577bfc240ec03e1d94/arch/x86/mm/tlb.c
,
Nov 10 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromiumos/third_party/kernel/+/b61be5eac47183e7511a2745aca4f1d532bb5012 commit b61be5eac47183e7511a2745aca4f1d532bb5012 Author: Steven Rostedt <rostedt@goodmis.org> Date: Fri Nov 10 21:13:54 2017 UPSTREAM: ARM: Hide finish_arch_post_lock_switch() from modules The introduction of switch_mm_irqs_off() brought back an old bug regarding the use of preempt_enable_no_resched: As part of: 62b94a08da1b ("sched/preempt: Take away preempt_enable_no_resched() from modules") the definition of preempt_enable_no_resched() is only available in built-in code, not in loadable modules, so we can't generally use it from header files. However, the ARM version of finish_arch_post_lock_switch() calls preempt_enable_no_resched() and is defined as a static inline function in asm/mmu_context.h. This in turn means we cannot include asm/mmu_context.h from modules. With today's tip tree, asm/mmu_context.h gets included from linux/mmu_context.h, which is normally the exact pattern one would expect, but unfortunately, linux/mmu_context.h can be included from the vhost driver that is a loadable module, now causing this compile time error with modular configs: In file included from ../include/linux/mmu_context.h:4:0, from ../drivers/vhost/vhost.c:18: ../arch/arm/include/asm/mmu_context.h: In function 'finish_arch_post_lock_switch': ../arch/arm/include/asm/mmu_context.h:88:3: error: implicit declaration of function 'preempt_enable_no_resched' [-Werror=implicit-function-declaration] preempt_enable_no_resched(); Andy already tried to fix the bug by including linux/preempt.h from asm/mmu_context.h, but that didn't help. Arnd suggested reordering the header files, which wasn't popular, so let's use this workaround instead: The finish_arch_post_lock_switch() definition is now also hidden inside of #ifdef MODULE, so we don't see anything referencing preempt_enable_no_resched() from a header file. I've built a few hundred randconfig kernels with this, and did not see any new problems. BUG=chromium:779077 TEST=Build, boot on soraka. Tested-by: Guenter Roeck <linux@roeck-us.net> Signed-off-by: Steven Rostedt <rostedt@goodmis.org> Signed-off-by: Arnd Bergmann <arnd@arndb.de> Acked-by: Russell King <rmk+kernel@arm.linux.org.uk> Cc: Alexander Shishkin <alexander.shishkin@linux.intel.com> Cc: Andy Lutomirski <luto@amacapital.net> Cc: Andy Lutomirski <luto@kernel.org> Cc: Ard Biesheuvel <ard.biesheuvel@linaro.org> Cc: Arnaldo Carvalho de Melo <acme@redhat.com> Cc: Borislav Petkov <bp@suse.de> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Linus Torvalds <torvalds@linux-foundation.org> Cc: Mel Gorman <mgorman@techsingularity.net> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Russell King - ARM Linux <linux@armlinux.org.uk> Cc: Stephane Eranian <eranian@google.com> Cc: Thomas Gleixner <tglx@linutronix.de> Cc: Vince Weaver <vincent.weaver@maine.edu> Cc: linux-arm-kernel@lists.infradead.org Fixes: f98db6013c55 ("sched/core: Add switch_mm_irqs_off() and use it in the scheduler") Link: http://lkml.kernel.org/r/1463146234-161304-1-git-send-email-arnd@arndb.de Signed-off-by: Ingo Molnar <mingo@kernel.org> (cherry picked from commit ef0491ea17f8019821c7e9c8e801184ecf17f85a) Change-Id: I01861e082dea345e46152b2a06765e9634ac84fa Reviewed-on: https://chromium-review.googlesource.com/741906 Commit-Ready: Jorge Lucangeli Obes <jorgelo@chromium.org> Tested-by: Jorge Lucangeli Obes <jorgelo@chromium.org> Reviewed-by: Guenter Roeck <groeck@chromium.org> (cherry picked from commit 71cf954e4cf0c146a94a557dc2c6877f300928b1) Signed-off-by: Aditya Kali <adityakali@google.com> Reviewed-on: https://chromium-review.googlesource.com/762055 Reviewed-by: Jorge Lucangeli Obes <jorgelo@chromium.org> Commit-Queue: Jorge Lucangeli Obes <jorgelo@chromium.org> [modify] https://crrev.com/b61be5eac47183e7511a2745aca4f1d532bb5012/arch/arm/include/asm/mmu_context.h
,
Nov 10 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromiumos/third_party/kernel/+/2a82b987380977a740d6416d524556361ceb7e44 commit 2a82b987380977a740d6416d524556361ceb7e44 Author: Andy Lutomirski <luto@kernel.org> Date: Fri Nov 10 21:13:58 2017 UPSTREAM: sched/core: Idle_task_exit() shouldn't use switch_mm_irqs_off() idle_task_exit() can be called with IRQs on x86 on and therefore should use switch_mm(), not switch_mm_irqs_off(). This doesn't seem to cause any problems right now, but it will confuse my upcoming TLB flush changes. Nonetheless, I think it should be backported because it's trivial. There won't be any meaningful performance impact because idle_task_exit() is only used when offlining a CPU. BUG=chromium:779077 TEST=Build, boot on soraka. Signed-off-by: Andy Lutomirski <luto@kernel.org> Cc: Borislav Petkov <bp@suse.de> Cc: Linus Torvalds <torvalds@linux-foundation.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Thomas Gleixner <tglx@linutronix.de> Cc: stable@vger.kernel.org Fixes: f98db6013c55 ("sched/core: Add switch_mm_irqs_off() and use it in the scheduler") Link: http://lkml.kernel.org/r/ca3d1a9fa93a0b49f5a8ff729eda3640fb6abdf9.1497034141.git.luto@kernel.org Signed-off-by: Ingo Molnar <mingo@kernel.org> (cherry picked from commit 252d2a4117bc181b287eeddf848863788da733ae) Change-Id: I65b268b1c8654b0df554abdca3d2290fba8a94b3 Reviewed-on: https://chromium-review.googlesource.com/741907 Commit-Ready: Jorge Lucangeli Obes <jorgelo@chromium.org> Tested-by: Jorge Lucangeli Obes <jorgelo@chromium.org> Reviewed-by: Guenter Roeck <groeck@chromium.org> (cherry picked from commit 58ee5e48938005bbcb31776b59a7858db3813238) Signed-off-by: Aditya Kali <adityakali@google.com> Reviewed-on: https://chromium-review.googlesource.com/762747 Reviewed-by: Jorge Lucangeli Obes <jorgelo@chromium.org> Commit-Queue: Jorge Lucangeli Obes <jorgelo@chromium.org> [modify] https://crrev.com/2a82b987380977a740d6416d524556361ceb7e44/kernel/sched/core.c
,
Nov 10 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromiumos/third_party/kernel/+/c32ee5175ecb7a037198d5aaf4ae5b28058a29ec commit c32ee5175ecb7a037198d5aaf4ae5b28058a29ec Author: Aaron Lu <aaron.lu@intel.com> Date: Fri Nov 10 21:14:02 2017 UPSTREAM: x86/irq: Do not substract irq_tlb_count from irq_call_count Since commit: 52aec3308db8 ("x86/tlb: replace INVALIDATE_TLB_VECTOR by CALL_FUNCTION_VECTOR") the TLB remote shootdown is done through call function vector. That commit didn't take care of irq_tlb_count, which a later commit: fd0f5869724f ("x86: Distinguish TLB shootdown interrupts from other functions call interrupts") ... tried to fix. The fix assumes every increase of irq_tlb_count has a corresponding increase of irq_call_count. So the irq_call_count is always bigger than irq_tlb_count and we could substract irq_tlb_count from irq_call_count. Unfortunately this is not true for the smp_call_function_single() case. The IPI is only sent if the target CPU's call_single_queue is empty when adding a csd into it in generic_exec_single. That means if two threads are both adding flush tlb csds to the same CPU's call_single_queue, only one IPI is sent. In other words, the irq_call_count is incremented by 1 but irq_tlb_count is incremented by 2. Over time, irq_tlb_count will be bigger than irq_call_count and the substract will produce a very large irq_call_count value due to overflow. Considering that: 1) it's not worth to send more IPIs for the sake of accurate counting of irq_call_count in generic_exec_single(); 2) it's not easy to tell if the call function interrupt is for TLB shootdown in __smp_call_function_single_interrupt(). Not to exclude TLB shootdown from call function count seems to be the simplest fix and this patch just does that. This bug was found by LKP's cyclic performance regression tracking recently with the vm-scalability test suite. I have bisected to commit: 3dec0ba0be6a ("mm/rmap: share the i_mmap_rwsem") This commit didn't do anything wrong but revealed the irq_call_count problem. IIUC, the commit makes rwc->remap_one in rmap_walk_file concurrent with multiple threads. When remap_one is try_to_unmap_one(), then multiple threads could queue flush TLB to the same CPU but only one IPI will be sent. Since the commit was added in Linux v3.19, the counting problem only shows up from v3.19 onwards. BUG=chromium:779077 TEST=Build, boot on soraka. Signed-off-by: Aaron Lu <aaron.lu@intel.com> Cc: Alex Shi <alex.shi@linaro.org> Cc: Andy Lutomirski <luto@kernel.org> Cc: Borislav Petkov <bp@alien8.de> Cc: Brian Gerst <brgerst@gmail.com> Cc: Davidlohr Bueso <dave@stgolabs.net> Cc: Denys Vlasenko <dvlasenk@redhat.com> Cc: H. Peter Anvin <hpa@zytor.com> Cc: Huang Ying <ying.huang@intel.com> Cc: Josh Poimboeuf <jpoimboe@redhat.com> Cc: Linus Torvalds <torvalds@linux-foundation.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Thomas Gleixner <tglx@linutronix.de> Cc: Tomoki Sekiyama <tomoki.sekiyama.qu@hitachi.com> Link: http://lkml.kernel.org/r/20160811074430.GA18163@aaronlu.sh.intel.com Signed-off-by: Ingo Molnar <mingo@kernel.org> (cherry picked from commit 82ba4faca1bffad429f15c90c980ffd010366c25) Change-Id: Id2795ea61eaa127bf90b2d06ee8fe1c01fd1ca5b Reviewed-on: https://chromium-review.googlesource.com/741908 Commit-Ready: Jorge Lucangeli Obes <jorgelo@chromium.org> Tested-by: Jorge Lucangeli Obes <jorgelo@chromium.org> Reviewed-by: Guenter Roeck <groeck@chromium.org> (cherry picked from commit 7c99f796caea737c792c7ad4aba2cab9cfa929f1) Signed-off-by: Aditya Kali <adityakali@google.com> Reviewed-on: https://chromium-review.googlesource.com/762748 Reviewed-by: Jorge Lucangeli Obes <jorgelo@chromium.org> Commit-Queue: Jorge Lucangeli Obes <jorgelo@chromium.org> [modify] https://crrev.com/c32ee5175ecb7a037198d5aaf4ae5b28058a29ec/arch/x86/kernel/irq.c [modify] https://crrev.com/c32ee5175ecb7a037198d5aaf4ae5b28058a29ec/arch/x86/include/asm/hardirq.h
,
Nov 10 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromiumos/third_party/kernel/+/78c0f2f39e551bf37b64efa204f77ad14dd1bc10 commit 78c0f2f39e551bf37b64efa204f77ad14dd1bc10 Author: Andy Lutomirski <luto@kernel.org> Date: Fri Nov 10 21:14:16 2017 UPSTREAM: x86/vm86/32: Switch to flush_tlb_mm_range() in mark_screen_rdonly() mark_screen_rdonly() is the last remaining caller of flush_tlb(). flush_tlb_mm_range() is potentially faster and isn't obsolete. Compile-tested only because I don't know whether software that uses this mechanism even exists. BUG=chromium:779077 TEST=Build, boot on soraka. Signed-off-by: Andy Lutomirski <luto@kernel.org> Cc: Andrew Morton <akpm@linux-foundation.org> Cc: Borislav Petkov <bp@alien8.de> Cc: Brian Gerst <brgerst@gmail.com> Cc: Dave Hansen <dave.hansen@intel.com> Cc: Denys Vlasenko <dvlasenk@redhat.com> Cc: H. Peter Anvin <hpa@zytor.com> Cc: Josh Poimboeuf <jpoimboe@redhat.com> Cc: Linus Torvalds <torvalds@linux-foundation.org> Cc: Michal Hocko <mhocko@suse.com> Cc: Nadav Amit <namit@vmware.com> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Rik van Riel <riel@redhat.com> Cc: Sasha Levin <sasha.levin@oracle.com> Cc: Thomas Gleixner <tglx@linutronix.de> Link: http://lkml.kernel.org/r/791a644076fc3577ba7f7b7cafd643cc089baa7d.1492844372.git.luto@kernel.org Signed-off-by: Ingo Molnar <mingo@kernel.org> (cherry picked from commit 9ccee2373f0658f234727700e619df097ba57023) Change-Id: I1a1e0f9276cc0856624f6af434c050ae36c78757 Reviewed-on: https://chromium-review.googlesource.com/741909 Commit-Ready: Jorge Lucangeli Obes <jorgelo@chromium.org> Tested-by: Jorge Lucangeli Obes <jorgelo@chromium.org> Reviewed-by: Guenter Roeck <groeck@chromium.org> (cherry picked from commit 48b90532c61b8b1421a8756032f885b651e9574b) Signed-off-by: Aditya Kali <adityakali@google.com> Reviewed-on: https://chromium-review.googlesource.com/762749 Reviewed-by: Jorge Lucangeli Obes <jorgelo@chromium.org> Commit-Queue: Jorge Lucangeli Obes <jorgelo@chromium.org> [modify] https://crrev.com/78c0f2f39e551bf37b64efa204f77ad14dd1bc10/arch/x86/kernel/vm86_32.c
,
Nov 10 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromiumos/third_party/kernel/+/454a20fa64b2331ef973e695d2a4f4e9df1c5bb0 commit 454a20fa64b2331ef973e695d2a4f4e9df1c5bb0 Author: Andy Lutomirski <luto@kernel.org> Date: Fri Nov 10 21:14:21 2017 UPSTREAM: x86/mm: Remove flush_tlb() and flush_tlb_current_task() I was trying to figure out what how flush_tlb_current_task() would possibly work correctly if current->mm != current->active_mm, but I realized I could spare myself the effort: it has no callers except the unused flush_tlb() macro. BUG=chromium:779077 TEST=Build, boot on soraka. Signed-off-by: Andy Lutomirski <luto@kernel.org> Cc: Andrew Morton <akpm@linux-foundation.org> Cc: Borislav Petkov <bp@alien8.de> Cc: Brian Gerst <brgerst@gmail.com> Cc: Dave Hansen <dave.hansen@intel.com> Cc: Denys Vlasenko <dvlasenk@redhat.com> Cc: H. Peter Anvin <hpa@zytor.com> Cc: Josh Poimboeuf <jpoimboe@redhat.com> Cc: Linus Torvalds <torvalds@linux-foundation.org> Cc: Michal Hocko <mhocko@suse.com> Cc: Nadav Amit <namit@vmware.com> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Rik van Riel <riel@redhat.com> Cc: Thomas Gleixner <tglx@linutronix.de> Link: http://lkml.kernel.org/r/e52d64c11690f85e9f1d69d7b48cc2269cd2e94b.1492844372.git.luto@kernel.org Signed-off-by: Ingo Molnar <mingo@kernel.org> (cherry picked from commit 29961b59a51f8c6838a26a45e871a7ed6771809b) Change-Id: Ie60b99e6c492e5557190c9d730143b90a667ece3 Reviewed-on: https://chromium-review.googlesource.com/741910 Commit-Ready: Jorge Lucangeli Obes <jorgelo@chromium.org> Tested-by: Jorge Lucangeli Obes <jorgelo@chromium.org> Reviewed-by: Guenter Roeck <groeck@chromium.org> (cherry picked from commit 583dc8e9bfed2d15102deb4ad105872d9c410020) Signed-off-by: Aditya Kali <adityakali@google.com> Reviewed-on: https://chromium-review.googlesource.com/762750 Reviewed-by: Jorge Lucangeli Obes <jorgelo@chromium.org> Commit-Queue: Jorge Lucangeli Obes <jorgelo@chromium.org> [modify] https://crrev.com/454a20fa64b2331ef973e695d2a4f4e9df1c5bb0/arch/x86/include/asm/tlbflush.h [modify] https://crrev.com/454a20fa64b2331ef973e695d2a4f4e9df1c5bb0/arch/x86/mm/tlb.c
,
Nov 10 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromiumos/third_party/kernel/+/7cc9245721f52d517552981d488f737059d92718 commit 7cc9245721f52d517552981d488f737059d92718 Author: Andy Lutomirski <luto@kernel.org> Date: Fri Nov 10 21:14:24 2017 UPSTREAM: x86/mm: Make flush_tlb_mm_range() more predictable I'm about to rewrite the function almost completely, but first I want to get a functional change out of the way. Currently, if flush_tlb_mm_range() does not flush the local TLB at all, it will never do individual page flushes on remote CPUs. This seems to be an accident, and preserving it will be awkward. Let's change it first so that any regressions in the rewrite will be easier to bisect and so that the rewrite can attempt to change no visible behavior at all. The fix is simple: we can simply avoid short-circuiting the calculation of base_pages_to_flush. As a side effect, this also eliminates a potential corner case: if tlb_single_page_flush_ceiling == TLB_FLUSH_ALL, flush_tlb_mm_range() could have ended up flushing the entire address space one page at a time. BUG=chromium:779077 TEST=Build, boot on soraka. Signed-off-by: Andy Lutomirski <luto@kernel.org> Acked-by: Dave Hansen <dave.hansen@intel.com> Cc: Andrew Morton <akpm@linux-foundation.org> Cc: Borislav Petkov <bp@alien8.de> Cc: Brian Gerst <brgerst@gmail.com> Cc: Denys Vlasenko <dvlasenk@redhat.com> Cc: H. Peter Anvin <hpa@zytor.com> Cc: Josh Poimboeuf <jpoimboe@redhat.com> Cc: Linus Torvalds <torvalds@linux-foundation.org> Cc: Michal Hocko <mhocko@suse.com> Cc: Nadav Amit <namit@vmware.com> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Rik van Riel <riel@redhat.com> Cc: Thomas Gleixner <tglx@linutronix.de> Link: http://lkml.kernel.org/r/4b29b771d9975aad7154c314534fec235618175a.1492844372.git.luto@kernel.org Signed-off-by: Ingo Molnar <mingo@kernel.org> (cherry picked from commit ce27374fabf553153c3f53efcaa9bfab9216bd8c) Change-Id: If0567ae7a6c629e332605f907af7f11332106bef Reviewed-on: https://chromium-review.googlesource.com/741911 Commit-Ready: Jorge Lucangeli Obes <jorgelo@chromium.org> Tested-by: Jorge Lucangeli Obes <jorgelo@chromium.org> Reviewed-by: Guenter Roeck <groeck@chromium.org> (cherry picked from commit d8a45e7a6989d7b6e1302c254888aeb560a1103c) Signed-off-by: Aditya Kali <adityakali@google.com> Reviewed-on: https://chromium-review.googlesource.com/762751 Reviewed-by: Jorge Lucangeli Obes <jorgelo@chromium.org> Commit-Queue: Jorge Lucangeli Obes <jorgelo@chromium.org> [modify] https://crrev.com/7cc9245721f52d517552981d488f737059d92718/arch/x86/mm/tlb.c
,
Nov 10 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromiumos/third_party/kernel/+/d3cce2e1cf1ec6d8ca8e9510ac582a9316ad45ab commit d3cce2e1cf1ec6d8ca8e9510ac582a9316ad45ab Author: Andy Lutomirski <luto@kernel.org> Date: Fri Nov 10 21:14:29 2017 UPSTREAM: x86/mm: Reimplement flush_tlb_page() using flush_tlb_mm_range() flush_tlb_page() was very similar to flush_tlb_mm_range() except that it had a couple of issues: - It was missing an smp_mb() in the case where current->active_mm != mm. (This is a longstanding bug reported by Nadav Amit) - It was missing tracepoints and vm counter updates. The only reason that I can see for keeping it at as a separate function is that it could avoid a few branches that flush_tlb_mm_range() needs to decide to flush just one page. This hardly seems worthwhile. If we decide we want to get rid of those branches again, a better way would be to introduce an __flush_tlb_mm_range() helper and make both flush_tlb_page() and flush_tlb_mm_range() use it. BUG=chromium:779077 TEST=Build, boot on soraka. Signed-off-by: Andy Lutomirski <luto@kernel.org> Acked-by: Kees Cook <keescook@chromium.org> Cc: Andrew Morton <akpm@linux-foundation.org> Cc: Borislav Petkov <bpetkov@suse.de> Cc: Dave Hansen <dave.hansen@intel.com> Cc: Linus Torvalds <torvalds@linux-foundation.org> Cc: Mel Gorman <mgorman@suse.de> Cc: Michal Hocko <mhocko@suse.com> Cc: Nadav Amit <nadav.amit@gmail.com> Cc: Nadav Amit <namit@vmware.com> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Rik van Riel <riel@redhat.com> Cc: Thomas Gleixner <tglx@linutronix.de> Cc: linux-mm@kvack.org Link: http://lkml.kernel.org/r/3cc3847cf888d8907577569b8bac3f01992ef8f9.1495492063.git.luto@kernel.org Signed-off-by: Ingo Molnar <mingo@kernel.org> (cherry picked from commit ca6c99c0794875c6d1db6e22f246699691ab7e6b) Change-Id: I681666b8c6d94a8d6d1aa177136fafc206c28ccb Reviewed-on: https://chromium-review.googlesource.com/741912 Commit-Ready: Jorge Lucangeli Obes <jorgelo@chromium.org> Tested-by: Jorge Lucangeli Obes <jorgelo@chromium.org> Reviewed-by: Guenter Roeck <groeck@chromium.org> (cherry picked from commit 1b53b4d133ea4dcff396590fb28262e930d694cb) Signed-off-by: Aditya Kali <adityakali@google.com> Reviewed-on: https://chromium-review.googlesource.com/762752 Reviewed-by: Jorge Lucangeli Obes <jorgelo@chromium.org> Commit-Queue: Jorge Lucangeli Obes <jorgelo@chromium.org> [modify] https://crrev.com/d3cce2e1cf1ec6d8ca8e9510ac582a9316ad45ab/arch/x86/include/asm/tlbflush.h [modify] https://crrev.com/d3cce2e1cf1ec6d8ca8e9510ac582a9316ad45ab/arch/x86/mm/tlb.c
,
Nov 10 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromiumos/third_party/kernel/+/6943fc7bac37206e5d2fb74fe7b109e7a6c35f76 commit 6943fc7bac37206e5d2fb74fe7b109e7a6c35f76 Author: Andy Lutomirski <luto@kernel.org> Date: Fri Nov 10 21:14:34 2017 BACKPORT: x86/mm: Remove the UP asm/tlbflush.h code, always use the (formerly) SMP code The UP asm/tlbflush.h generates somewhat nicer code than the SMP version. Aside from that, it's fallen quite a bit behind the SMP code: - flush_tlb_mm_range() didn't flush individual pages if the range was small. - The lazy TLB code was much weaker. This usually wouldn't matter, but, if a kernel thread flushed its lazy "active_mm" more than once (due to reclaim or similar), it wouldn't be unlazied and would instead pointlessly flush repeatedly. - Tracepoints were missing. Aside from that, simply having the UP code around was a maintanence burden, since it means that any change to the TLB flush code had to make sure not to break it. Simplify everything by deleting the UP code. BUG=chromium:779077 TEST=Build, boot on soraka. Signed-off-by: Andy Lutomirski <luto@kernel.org> Cc: Andrew Morton <akpm@linux-foundation.org> Cc: Arjan van de Ven <arjan@linux.intel.com> Cc: Borislav Petkov <bpetkov@suse.de> Cc: Dave Hansen <dave.hansen@intel.com> Cc: Linus Torvalds <torvalds@linux-foundation.org> Cc: Mel Gorman <mgorman@suse.de> Cc: Michal Hocko <mhocko@suse.com> Cc: Nadav Amit <nadav.amit@gmail.com> Cc: Nadav Amit <namit@vmware.com> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Rik van Riel <riel@redhat.com> Cc: Thomas Gleixner <tglx@linutronix.de> Cc: linux-mm@kvack.org Signed-off-by: Ingo Molnar <mingo@kernel.org> (cherry picked from commit ce4a4e565f5264909a18c733b864c3f74467f69e) Conflicts: arch/x86/Kconfig arch/x86/include/asm/tlbbatch.h arch/x86/include/asm/tlbflush.h arch/x86/mm/tlb.c Change-Id: I71687765e75ac212e7eea239c5373aabf7b2bfbf Reviewed-on: https://chromium-review.googlesource.com/741913 Commit-Ready: Jorge Lucangeli Obes <jorgelo@chromium.org> Tested-by: Jorge Lucangeli Obes <jorgelo@chromium.org> Reviewed-by: Guenter Roeck <groeck@chromium.org> (cherry picked from commit b7c0e9d3714dc250d920264e8a9bddf3f701c4aa) Signed-off-by: Aditya Kali <adityakali@google.com> Reviewed-on: https://chromium-review.googlesource.com/762753 Reviewed-by: Jorge Lucangeli Obes <jorgelo@chromium.org> Commit-Queue: Jorge Lucangeli Obes <jorgelo@chromium.org> [modify] https://crrev.com/6943fc7bac37206e5d2fb74fe7b109e7a6c35f76/arch/x86/mm/tlb.c [modify] https://crrev.com/6943fc7bac37206e5d2fb74fe7b109e7a6c35f76/arch/x86/include/asm/mmu_context.h [modify] https://crrev.com/6943fc7bac37206e5d2fb74fe7b109e7a6c35f76/arch/x86/include/asm/hardirq.h [modify] https://crrev.com/6943fc7bac37206e5d2fb74fe7b109e7a6c35f76/arch/x86/mm/init.c [modify] https://crrev.com/6943fc7bac37206e5d2fb74fe7b109e7a6c35f76/arch/x86/include/asm/tlbflush.h [modify] https://crrev.com/6943fc7bac37206e5d2fb74fe7b109e7a6c35f76/arch/x86/Kconfig [modify] https://crrev.com/6943fc7bac37206e5d2fb74fe7b109e7a6c35f76/arch/x86/include/asm/mmu.h
,
Nov 10 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromiumos/third_party/kernel/+/7d1a3736cd183455cb41504fff33921c992ff65e commit 7d1a3736cd183455cb41504fff33921c992ff65e Author: Andy Lutomirski <luto@kernel.org> Date: Fri Nov 10 21:14:38 2017 UPSTREAM: x86/mm: Disable PCID on 32-bit kernels 32-bit kernels on new hardware will see PCID in CPUID, but PCID can only be used in 64-bit mode. Rather than making all PCID code conditional, just disable the feature on 32-bit builds. BUG=chromium:779077 TEST=Build, boot on soraka. Signed-off-by: Andy Lutomirski <luto@kernel.org> Reviewed-by: Nadav Amit <nadav.amit@gmail.com> Reviewed-by: Borislav Petkov <bp@suse.de> Reviewed-by: Thomas Gleixner <tglx@linutronix.de> Cc: Andrew Morton <akpm@linux-foundation.org> Cc: Arjan van de Ven <arjan@linux.intel.com> Cc: Borislav Petkov <bp@alien8.de> Cc: Dave Hansen <dave.hansen@intel.com> Cc: Linus Torvalds <torvalds@linux-foundation.org> Cc: Mel Gorman <mgorman@suse.de> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Rik van Riel <riel@redhat.com> Cc: linux-mm@kvack.org Link: http://lkml.kernel.org/r/2e391769192a4d31b808410c383c6bf0734bc6ea.1498751203.git.luto@kernel.org Signed-off-by: Ingo Molnar <mingo@kernel.org> (cherry picked from commit cba4671af7550e008f7a7835f06df0763825bf3e) Change-Id: I2f91901f636958a0e19e2cfd02e7d6e143a9d3d0 Reviewed-on: https://chromium-review.googlesource.com/741914 Commit-Ready: Jorge Lucangeli Obes <jorgelo@chromium.org> Tested-by: Jorge Lucangeli Obes <jorgelo@chromium.org> Reviewed-by: Guenter Roeck <groeck@chromium.org> (cherry picked from commit 5929d58eab3898fdb995c77d339930498a3b926d) Signed-off-by: Aditya Kali <adityakali@google.com> Reviewed-on: https://chromium-review.googlesource.com/762754 Reviewed-by: Jorge Lucangeli Obes <jorgelo@chromium.org> Commit-Queue: Jorge Lucangeli Obes <jorgelo@chromium.org> [modify] https://crrev.com/7d1a3736cd183455cb41504fff33921c992ff65e/arch/x86/include/asm/disabled-features.h [modify] https://crrev.com/7d1a3736cd183455cb41504fff33921c992ff65e/arch/x86/kernel/cpu/bugs.c
,
Nov 10 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromiumos/third_party/kernel/+/8307f22c9aba55239207d1777b5545d502c4f3c3 commit 8307f22c9aba55239207d1777b5545d502c4f3c3 Author: Andy Lutomirski <luto@kernel.org> Date: Fri Nov 10 21:14:42 2017 BACKPORT: x86/mm: Add the 'nopcid' boot option to turn off PCID The parameter is only present on x86_64 systems to save a few bytes, as PCID is always disabled on x86_32. BUG=chromium:779077 TEST=Build, boot on soraka. Signed-off-by: Andy Lutomirski <luto@kernel.org> Reviewed-by: Nadav Amit <nadav.amit@gmail.com> Reviewed-by: Borislav Petkov <bp@suse.de> Reviewed-by: Thomas Gleixner <tglx@linutronix.de> Cc: Andrew Morton <akpm@linux-foundation.org> Cc: Arjan van de Ven <arjan@linux.intel.com> Cc: Borislav Petkov <bp@alien8.de> Cc: Dave Hansen <dave.hansen@intel.com> Cc: Linus Torvalds <torvalds@linux-foundation.org> Cc: Mel Gorman <mgorman@suse.de> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Rik van Riel <riel@redhat.com> Cc: linux-mm@kvack.org Link: http://lkml.kernel.org/r/8bbb2e65bcd249a5f18bfb8128b4689f08ac2b60.1498751203.git.luto@kernel.org Signed-off-by: Ingo Molnar <mingo@kernel.org> (cherry picked from commit 0790c9aad84901ca1bdc14746175549c8b5da215) Conflicts: Documentation/admin-guide/kernel-parameters.txt (not in this tree) Documentation/kernel-parameters.txt (patched instead of that) Change-Id: I439569f5b200a1d3d04e46ac0e0b4d49aeefe5f9 Reviewed-on: https://chromium-review.googlesource.com/741915 Commit-Ready: Jorge Lucangeli Obes <jorgelo@chromium.org> Tested-by: Jorge Lucangeli Obes <jorgelo@chromium.org> Reviewed-by: Guenter Roeck <groeck@chromium.org> (cherry picked from commit d0b2a4feb02ae78a3dcc621a0e126feb417cb82a) Signed-off-by: Aditya Kali <adityakali@google.com> Reviewed-on: https://chromium-review.googlesource.com/762755 Reviewed-by: Jorge Lucangeli Obes <jorgelo@chromium.org> Commit-Queue: Jorge Lucangeli Obes <jorgelo@chromium.org> [modify] https://crrev.com/8307f22c9aba55239207d1777b5545d502c4f3c3/Documentation/kernel-parameters.txt [modify] https://crrev.com/8307f22c9aba55239207d1777b5545d502c4f3c3/arch/x86/kernel/cpu/common.c
,
Nov 10 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromiumos/third_party/kernel/+/0b76c8673dd956e632ffb226484dd4d9140ae9c5 commit 0b76c8673dd956e632ffb226484dd4d9140ae9c5 Author: Andy Lutomirski <luto@kernel.org> Date: Fri Nov 10 21:14:46 2017 BACKPORT: x86/mm: Enable CR4.PCIDE on supported systems We can use PCID if the CPU has PCID and PGE and we're not on Xen. By itself, this has no effect. A followup patch will start using PCID. BUG=chromium:779077 TEST=Build, boot on soraka. Signed-off-by: Andy Lutomirski <luto@kernel.org> Reviewed-by: Nadav Amit <nadav.amit@gmail.com> Reviewed-by: Boris Ostrovsky <boris.ostrovsky@oracle.com> Reviewed-by: Thomas Gleixner <tglx@linutronix.de> Cc: Andrew Morton <akpm@linux-foundation.org> Cc: Arjan van de Ven <arjan@linux.intel.com> Cc: Borislav Petkov <bp@alien8.de> Cc: Dave Hansen <dave.hansen@intel.com> Cc: Juergen Gross <jgross@suse.com> Cc: Linus Torvalds <torvalds@linux-foundation.org> Cc: Mel Gorman <mgorman@suse.de> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Rik van Riel <riel@redhat.com> Cc: linux-mm@kvack.org Link: http://lkml.kernel.org/r/6327ecd907b32f79d5aa0d466f04503bbec5df88.1498751203.git.luto@kernel.org Signed-off-by: Ingo Molnar <mingo@kernel.org> (cherry picked from commit 660da7c9228f685b2ebe664f9fd69aaddcc420b5) Conflicts: arch/x86/xen/enlighten_pv.c (not in this tree) arch/x86/xen/enlighten.c (patched instead of that) Change-Id: I4a0991e84f87d875b796981c73d87aee5f452ee0 Reviewed-on: https://chromium-review.googlesource.com/741916 Commit-Ready: Jorge Lucangeli Obes <jorgelo@chromium.org> Tested-by: Jorge Lucangeli Obes <jorgelo@chromium.org> Reviewed-by: Guenter Roeck <groeck@chromium.org> (cherry picked from commit 82a9a43487eeb9ef47ad13a7d314de35c68e9199) Signed-off-by: Aditya Kali <adityakali@google.com> Reviewed-on: https://chromium-review.googlesource.com/762756 Reviewed-by: Jorge Lucangeli Obes <jorgelo@chromium.org> Commit-Queue: Jorge Lucangeli Obes <jorgelo@chromium.org> [modify] https://crrev.com/0b76c8673dd956e632ffb226484dd4d9140ae9c5/arch/x86/include/asm/tlbflush.h [modify] https://crrev.com/0b76c8673dd956e632ffb226484dd4d9140ae9c5/arch/x86/xen/enlighten.c [modify] https://crrev.com/0b76c8673dd956e632ffb226484dd4d9140ae9c5/arch/x86/kernel/cpu/common.c
,
Nov 11 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromiumos/third_party/kernel/+/d54512ba4e37e579db43ac0422220404236b807b commit d54512ba4e37e579db43ac0422220404236b807b Author: Andy Lutomirski <luto@kernel.org> Date: Sat Nov 11 00:29:37 2017 UPSTREAM: x86/mm: Add INVPCID helpers This adds helpers for each of the four currently-specified INVPCID modes. BUG=chromium:779077 TEST=Build, boot on caroline. Signed-off-by: Andy Lutomirski <luto@kernel.org> Reviewed-by: Borislav Petkov <bp@suse.de> Cc: Andrew Morton <akpm@linux-foundation.org> Cc: Andrey Ryabinin <aryabinin@virtuozzo.com> Cc: Andy Lutomirski <luto@amacapital.net> Cc: Borislav Petkov <bp@alien8.de> Cc: Brian Gerst <brgerst@gmail.com> Cc: Dave Hansen <dave.hansen@linux.intel.com> Cc: Denys Vlasenko <dvlasenk@redhat.com> Cc: H. Peter Anvin <hpa@zytor.com> Cc: Linus Torvalds <torvalds@linux-foundation.org> Cc: Luis R. Rodriguez <mcgrof@suse.com> Cc: Oleg Nesterov <oleg@redhat.com> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Thomas Gleixner <tglx@linutronix.de> Cc: Toshi Kani <toshi.kani@hp.com> Cc: linux-mm@kvack.org Link: http://lkml.kernel.org/r/8a62b23ad686888cee01da134c91409e22064db9.1454096309.git.luto@kernel.org Signed-off-by: Ingo Molnar <mingo@kernel.org> (cherry picked from commit 060a402a1ddb551455ee410de2eadd3349f2801b) Change-Id: I49b2216bce7e3c8fe2a19dd068377bb7392f584f Reviewed-on: https://chromium-review.googlesource.com/742002 Commit-Ready: Jorge Lucangeli Obes <jorgelo@chromium.org> Tested-by: Jorge Lucangeli Obes <jorgelo@chromium.org> Reviewed-by: Guenter Roeck <groeck@chromium.org> Reviewed-on: https://chromium-review.googlesource.com/763921 Reviewed-by: Jorge Lucangeli Obes <jorgelo@chromium.org> Commit-Queue: Jorge Lucangeli Obes <jorgelo@chromium.org> [modify] https://crrev.com/d54512ba4e37e579db43ac0422220404236b807b/arch/x86/include/asm/tlbflush.h
,
Nov 11 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromiumos/third_party/kernel/+/37b9d02a4270cc33dac5cb62db5e3b46fd4a3813 commit 37b9d02a4270cc33dac5cb62db5e3b46fd4a3813 Author: Borislav Petkov <bp@suse.de> Date: Sat Nov 11 00:29:42 2017 UPSTREAM: x86/mm: Fix INVPCID asm constraint So we want to specify the dependency on both @pcid and @addr so that the compiler doesn't reorder accesses to them *before* the TLB flush. But for that to work, we need to express this properly in the inline asm and deref the whole desc array, not the pointer to it. See clwb() for an example. This fixes the build error on 32-bit: arch/x86/include/asm/tlbflush.h: In function __invpcid: arch/x86/include/asm/tlbflush.h:26:18: error: memory input 0 is not directly addressable which gcc4.7 caught but 5.x didn't. Which is strange. :-\ BUG=chromium:779077 TEST=Build, boot on caroline. Signed-off-by: Borislav Petkov <bp@suse.de> Cc: Andrew Morton <akpm@linux-foundation.org> Cc: Andrey Ryabinin <aryabinin@virtuozzo.com> Cc: Andy Lutomirski <luto@amacapital.net> Cc: Borislav Petkov <bp@alien8.de> Cc: Brian Gerst <brgerst@gmail.com> Cc: Dave Hansen <dave.hansen@linux.intel.com> Cc: Denys Vlasenko <dvlasenk@redhat.com> Cc: H. Peter Anvin <hpa@zytor.com> Cc: Linus Torvalds <torvalds@linux-foundation.org> Cc: Luis R. Rodriguez <mcgrof@suse.com> Cc: Michael Matz <matz@suse.de> Cc: Oleg Nesterov <oleg@redhat.com> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Thomas Gleixner <tglx@linutronix.de> Cc: Toshi Kani <toshi.kani@hp.com> Cc: linux-mm@kvack.org Signed-off-by: Ingo Molnar <mingo@kernel.org> (cherry picked from commit e2c7698cd61f11d4077fdb28148b2d31b82ac848) Change-Id: I428b1d1445f6756b7644869a5d323ef041799339 Reviewed-on: https://chromium-review.googlesource.com/742003 Commit-Ready: Jorge Lucangeli Obes <jorgelo@chromium.org> Tested-by: Jorge Lucangeli Obes <jorgelo@chromium.org> Reviewed-by: Guenter Roeck <groeck@chromium.org> Reviewed-on: https://chromium-review.googlesource.com/763922 Reviewed-by: Jorge Lucangeli Obes <jorgelo@chromium.org> Commit-Queue: Jorge Lucangeli Obes <jorgelo@chromium.org> [modify] https://crrev.com/37b9d02a4270cc33dac5cb62db5e3b46fd4a3813/arch/x86/include/asm/tlbflush.h
,
Nov 11 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromiumos/third_party/kernel/+/4997302036e7df823ed598ecce28793bfcf3a61c commit 4997302036e7df823ed598ecce28793bfcf3a61c Author: Andy Lutomirski <luto@kernel.org> Date: Sat Nov 11 00:29:45 2017 UPSTREAM: x86/mm: Add a 'noinvpcid' boot option to turn off INVPCID This adds a chicken bit to turn off INVPCID in case something goes wrong. It's an early_param() because we do TLB flushes before we parse __setup() parameters. BUG=chromium:779077 TEST=Build, boot on caroline. Signed-off-by: Andy Lutomirski <luto@kernel.org> Reviewed-by: Borislav Petkov <bp@suse.de> Cc: Andrew Morton <akpm@linux-foundation.org> Cc: Andrey Ryabinin <aryabinin@virtuozzo.com> Cc: Andy Lutomirski <luto@amacapital.net> Cc: Borislav Petkov <bp@alien8.de> Cc: Brian Gerst <brgerst@gmail.com> Cc: Dave Hansen <dave.hansen@linux.intel.com> Cc: Denys Vlasenko <dvlasenk@redhat.com> Cc: H. Peter Anvin <hpa@zytor.com> Cc: Linus Torvalds <torvalds@linux-foundation.org> Cc: Luis R. Rodriguez <mcgrof@suse.com> Cc: Oleg Nesterov <oleg@redhat.com> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Thomas Gleixner <tglx@linutronix.de> Cc: Toshi Kani <toshi.kani@hp.com> Cc: linux-mm@kvack.org Link: http://lkml.kernel.org/r/f586317ed1bc2b87aee652267e515b90051af385.1454096309.git.luto@kernel.org Signed-off-by: Ingo Molnar <mingo@kernel.org> (cherry picked from commit d12a72b844a49d4162f24cefdab30bed3f86730e) Change-Id: I259a8968ddcaca10edaa3fd7976a2a25d2afd4db Reviewed-on: https://chromium-review.googlesource.com/742004 Commit-Ready: Jorge Lucangeli Obes <jorgelo@chromium.org> Tested-by: Jorge Lucangeli Obes <jorgelo@chromium.org> Reviewed-by: Guenter Roeck <groeck@chromium.org> Reviewed-on: https://chromium-review.googlesource.com/763923 Reviewed-by: Jorge Lucangeli Obes <jorgelo@chromium.org> Commit-Queue: Jorge Lucangeli Obes <jorgelo@chromium.org> [modify] https://crrev.com/4997302036e7df823ed598ecce28793bfcf3a61c/Documentation/kernel-parameters.txt [modify] https://crrev.com/4997302036e7df823ed598ecce28793bfcf3a61c/arch/x86/kernel/cpu/common.c
,
Nov 11 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromiumos/third_party/kernel/+/f2b96db7823a3e0b8bdb0a44208b47fa386aa24f commit f2b96db7823a3e0b8bdb0a44208b47fa386aa24f Author: Andy Lutomirski <luto@kernel.org> Date: Sat Nov 11 00:29:49 2017 UPSTREAM: x86/mm: If INVPCID is available, use it to flush global mappings On my Skylake laptop, INVPCID function 2 (flush absolutely everything) takes about 376ns, whereas saving flags, twiddling CR4.PGE to flush global mappings, and restoring flags takes about 539ns. BUG=chromium:779077 TEST=Build, boot on caroline. Signed-off-by: Andy Lutomirski <luto@kernel.org> Reviewed-by: Borislav Petkov <bp@suse.de> Cc: Andrew Morton <akpm@linux-foundation.org> Cc: Andrey Ryabinin <aryabinin@virtuozzo.com> Cc: Andy Lutomirski <luto@amacapital.net> Cc: Borislav Petkov <bp@alien8.de> Cc: Brian Gerst <brgerst@gmail.com> Cc: Dave Hansen <dave.hansen@linux.intel.com> Cc: Denys Vlasenko <dvlasenk@redhat.com> Cc: H. Peter Anvin <hpa@zytor.com> Cc: Linus Torvalds <torvalds@linux-foundation.org> Cc: Luis R. Rodriguez <mcgrof@suse.com> Cc: Oleg Nesterov <oleg@redhat.com> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Thomas Gleixner <tglx@linutronix.de> Cc: Toshi Kani <toshi.kani@hp.com> Cc: linux-mm@kvack.org Link: http://lkml.kernel.org/r/ed0ef62581c0ea9c99b9bf6df726015e96d44743.1454096309.git.luto@kernel.org Signed-off-by: Ingo Molnar <mingo@kernel.org> (cherry picked from commit d8bced79af1db6734f66b42064cc773cada2ce99) Change-Id: I0a79ba87173caa1de1ce03dfd9d98258344103f3 Reviewed-on: https://chromium-review.googlesource.com/742005 Commit-Ready: Jorge Lucangeli Obes <jorgelo@chromium.org> Tested-by: Jorge Lucangeli Obes <jorgelo@chromium.org> Reviewed-by: Guenter Roeck <groeck@chromium.org> Reviewed-on: https://chromium-review.googlesource.com/763924 Reviewed-by: Jorge Lucangeli Obes <jorgelo@chromium.org> Commit-Queue: Jorge Lucangeli Obes <jorgelo@chromium.org> [modify] https://crrev.com/f2b96db7823a3e0b8bdb0a44208b47fa386aa24f/arch/x86/include/asm/tlbflush.h
,
Nov 11 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromiumos/third_party/kernel/+/365e7a70ddcc440c93138c2250f20aadb2efb894 commit 365e7a70ddcc440c93138c2250f20aadb2efb894 Author: Ingo Molnar <mingo@kernel.org> Date: Sat Nov 11 00:29:53 2017 UPSTREAM: mm/mmu_context, sched/core: Fix mmu_context.h assumption Some architectures (such as Alpha) rely on include/linux/sched.h definitions in their mmu_context.h files. So include sched.h before mmu_context.h. BUG=chromium:779077 TEST=Build, boot on caroline. Cc: Andy Lutomirski <luto@kernel.org> Cc: Borislav Petkov <bp@alien8.de> Cc: Linus Torvalds <torvalds@linux-foundation.org> Cc: linux-kernel@vger.kernel.org Cc: Peter Zijlstra <peterz@infradead.org> Cc: Thomas Gleixner <tglx@linutronix.de> Signed-off-by: Ingo Molnar <mingo@kernel.org> (cherry picked from commit 8efd755ac2fe262d4c8d5c9bbe054bb67dae93da) Change-Id: I2a4fcda74e6aa4576a899439fb35c065f86711b4 Reviewed-on: https://chromium-review.googlesource.com/742006 Commit-Ready: Jorge Lucangeli Obes <jorgelo@chromium.org> Tested-by: Jorge Lucangeli Obes <jorgelo@chromium.org> Reviewed-by: Guenter Roeck <groeck@chromium.org> Reviewed-on: https://chromium-review.googlesource.com/763925 Reviewed-by: Jorge Lucangeli Obes <jorgelo@chromium.org> Commit-Queue: Jorge Lucangeli Obes <jorgelo@chromium.org> [modify] https://crrev.com/365e7a70ddcc440c93138c2250f20aadb2efb894/mm/mmu_context.c
,
Nov 11 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromiumos/third_party/kernel/+/9eea1b7877c2c007393579a5ff32b68b2da6c66a commit 9eea1b7877c2c007393579a5ff32b68b2da6c66a Author: Andy Lutomirski <luto@kernel.org> Date: Sat Nov 11 00:29:57 2017 UPSTREAM: sched/core: Add switch_mm_irqs_off() and use it in the scheduler By default, this is the same thing as switch_mm(). x86 will override it as an optimization. BUG=chromium:779077 TEST=Build, boot on caroline. Signed-off-by: Andy Lutomirski <luto@kernel.org> Reviewed-by: Borislav Petkov <bp@suse.de> Cc: Borislav Petkov <bp@alien8.de> Cc: Linus Torvalds <torvalds@linux-foundation.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Thomas Gleixner <tglx@linutronix.de> Link: http://lkml.kernel.org/r/df401df47bdd6be3e389c6f1e3f5310d70e81b2c.1461688545.git.luto@kernel.org Signed-off-by: Ingo Molnar <mingo@kernel.org> (cherry picked from commit f98db6013c557c216da5038d9c52045be55cd039) Change-Id: I73dbc3409895bce461674be4480f8285acc35728 Reviewed-on: https://chromium-review.googlesource.com/742007 Commit-Ready: Jorge Lucangeli Obes <jorgelo@chromium.org> Tested-by: Jorge Lucangeli Obes <jorgelo@chromium.org> Reviewed-by: Guenter Roeck <groeck@chromium.org> Reviewed-on: https://chromium-review.googlesource.com/763926 Reviewed-by: Jorge Lucangeli Obes <jorgelo@chromium.org> Commit-Queue: Jorge Lucangeli Obes <jorgelo@chromium.org> [modify] https://crrev.com/9eea1b7877c2c007393579a5ff32b68b2da6c66a/include/linux/mmu_context.h [modify] https://crrev.com/9eea1b7877c2c007393579a5ff32b68b2da6c66a/kernel/sched/core.c
,
Nov 11 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromiumos/third_party/kernel/+/e3bf6aa33dc79d305c53dd3fd8db758634c48cbb commit e3bf6aa33dc79d305c53dd3fd8db758634c48cbb Author: Andy Lutomirski <luto@kernel.org> Date: Sat Nov 11 00:30:00 2017 BACKPORT: x86/mm: Build arch/x86/mm/tlb.c even on !SMP Currently all of the functions that live in tlb.c are inlined on !SMP builds. One can debate whether this is a good idea (in many respects the code in tlb.c is better than the inlined UP code). Regardless, I want to add code that needs to be built on UP and SMP kernels and relates to tlb flushing, so arrange for tlb.c to be compiled unconditionally. BUG=chromium:779077 TEST=Build, boot on caroline. Signed-off-by: Andy Lutomirski <luto@kernel.org> Reviewed-by: Borislav Petkov <bp@suse.de> Cc: Borislav Petkov <bp@alien8.de> Cc: Linus Torvalds <torvalds@linux-foundation.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Thomas Gleixner <tglx@linutronix.de> Link: http://lkml.kernel.org/r/f0d778f0d828fc46e5d1946bca80f0aaf9abf032.1461688545.git.luto@kernel.org Signed-off-by: Ingo Molnar <mingo@kernel.org> (cherry picked from commit e1074888c326038340a1ada9129d679e661f2ea6) Change-Id: I7649a2495ba7d47be2a379a086a574c1b6305aee Reviewed-on: https://chromium-review.googlesource.com/742008 Commit-Ready: Jorge Lucangeli Obes <jorgelo@chromium.org> Tested-by: Jorge Lucangeli Obes <jorgelo@chromium.org> Reviewed-by: Guenter Roeck <groeck@chromium.org> Reviewed-on: https://chromium-review.googlesource.com/764287 Reviewed-by: Jorge Lucangeli Obes <jorgelo@chromium.org> Commit-Queue: Jorge Lucangeli Obes <jorgelo@chromium.org> [modify] https://crrev.com/e3bf6aa33dc79d305c53dd3fd8db758634c48cbb/arch/x86/mm/tlb.c [modify] https://crrev.com/e3bf6aa33dc79d305c53dd3fd8db758634c48cbb/arch/x86/mm/Makefile
,
Nov 11 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromiumos/third_party/kernel/+/0688447e57d6fab5c0dea642860e9f3ddeb7f1be commit 0688447e57d6fab5c0dea642860e9f3ddeb7f1be Author: Andy Lutomirski <luto@kernel.org> Date: Sat Nov 11 00:30:04 2017 BACKPORT: x86/mm, sched/core: Uninline switch_mm() It's fairly large and it has quite a few callers. This may also help untangle some headers down the road. BUG=chromium:779077 TEST=Build, boot on caroline. Signed-off-by: Andy Lutomirski <luto@kernel.org> Reviewed-by: Borislav Petkov <bp@suse.de> Cc: Borislav Petkov <bp@alien8.de> Cc: Linus Torvalds <torvalds@linux-foundation.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Thomas Gleixner <tglx@linutronix.de> Link: http://lkml.kernel.org/r/54f3367803e7f80b2be62c8a21879aa74b1a5f57.1461688545.git.luto@kernel.org Signed-off-by: Ingo Molnar <mingo@kernel.org> (cherry picked from commit 69c0319aabba45bcf33178916a2f06967b4adede) Conflicts: arch/x86/include/asm/mmu_context.h Change-Id: I82a8ef651aabaaf9f9d7859a46d1fd24aca81a8c Reviewed-on: https://chromium-review.googlesource.com/742009 Commit-Ready: Jorge Lucangeli Obes <jorgelo@chromium.org> Tested-by: Jorge Lucangeli Obes <jorgelo@chromium.org> Reviewed-by: Guenter Roeck <groeck@chromium.org> Reviewed-on: https://chromium-review.googlesource.com/764288 Reviewed-by: Jorge Lucangeli Obes <jorgelo@chromium.org> Commit-Queue: Jorge Lucangeli Obes <jorgelo@chromium.org> [modify] https://crrev.com/0688447e57d6fab5c0dea642860e9f3ddeb7f1be/arch/x86/include/asm/mmu_context.h [modify] https://crrev.com/0688447e57d6fab5c0dea642860e9f3ddeb7f1be/arch/x86/mm/tlb.c
,
Nov 11 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromiumos/third_party/kernel/+/ffcb42177dc6471f3e3441024687913390c33bef commit ffcb42177dc6471f3e3441024687913390c33bef Author: Andy Lutomirski <luto@kernel.org> Date: Sat Nov 11 00:30:07 2017 UPSTREAM: x86/mm, sched/core: Turn off IRQs in switch_mm() Potential races between switch_mm() and TLB-flush or LDT-flush IPIs could be very messy. AFAICT the code is currently okay, whether by accident or by careful design, but enabling PCID will make it considerably more complicated and will no longer be obviously safe. Fix it with a big hammer: run switch_mm() with IRQs off. To avoid a performance hit in the scheduler, we take advantage of our knowledge that the scheduler already has IRQs disabled when it calls switch_mm(). BUG=chromium:779077 TEST=Build, boot on caroline. Signed-off-by: Andy Lutomirski <luto@kernel.org> Reviewed-by: Borislav Petkov <bp@suse.de> Cc: Borislav Petkov <bp@alien8.de> Cc: Linus Torvalds <torvalds@linux-foundation.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Thomas Gleixner <tglx@linutronix.de> Link: http://lkml.kernel.org/r/f19baf759693c9dcae64bbff76189db77cb13398.1461688545.git.luto@kernel.org Signed-off-by: Ingo Molnar <mingo@kernel.org> (cherry picked from commit 078194f8e9fe3cf54c8fd8bded48a1db5bd8eb8a) Change-Id: I2141b622df2cdb2bf09e4d7e0e8078d3c6434c2b Reviewed-on: https://chromium-review.googlesource.com/742010 Commit-Ready: Jorge Lucangeli Obes <jorgelo@chromium.org> Tested-by: Jorge Lucangeli Obes <jorgelo@chromium.org> Reviewed-by: Guenter Roeck <groeck@chromium.org> Reviewed-on: https://chromium-review.googlesource.com/764289 Reviewed-by: Jorge Lucangeli Obes <jorgelo@chromium.org> Commit-Queue: Jorge Lucangeli Obes <jorgelo@chromium.org> [modify] https://crrev.com/ffcb42177dc6471f3e3441024687913390c33bef/arch/x86/include/asm/mmu_context.h [modify] https://crrev.com/ffcb42177dc6471f3e3441024687913390c33bef/arch/x86/mm/tlb.c
,
Nov 11 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromiumos/third_party/kernel/+/d4f7d9d62e4151b68a025244cbcfc7922d861caf commit d4f7d9d62e4151b68a025244cbcfc7922d861caf Author: Steven Rostedt <rostedt@goodmis.org> Date: Sat Nov 11 00:30:11 2017 UPSTREAM: ARM: Hide finish_arch_post_lock_switch() from modules The introduction of switch_mm_irqs_off() brought back an old bug regarding the use of preempt_enable_no_resched: As part of: 62b94a08da1b ("sched/preempt: Take away preempt_enable_no_resched() from modules") the definition of preempt_enable_no_resched() is only available in built-in code, not in loadable modules, so we can't generally use it from header files. However, the ARM version of finish_arch_post_lock_switch() calls preempt_enable_no_resched() and is defined as a static inline function in asm/mmu_context.h. This in turn means we cannot include asm/mmu_context.h from modules. With today's tip tree, asm/mmu_context.h gets included from linux/mmu_context.h, which is normally the exact pattern one would expect, but unfortunately, linux/mmu_context.h can be included from the vhost driver that is a loadable module, now causing this compile time error with modular configs: In file included from ../include/linux/mmu_context.h:4:0, from ../drivers/vhost/vhost.c:18: ../arch/arm/include/asm/mmu_context.h: In function 'finish_arch_post_lock_switch': ../arch/arm/include/asm/mmu_context.h:88:3: error: implicit declaration of function 'preempt_enable_no_resched' [-Werror=implicit-function-declaration] preempt_enable_no_resched(); Andy already tried to fix the bug by including linux/preempt.h from asm/mmu_context.h, but that didn't help. Arnd suggested reordering the header files, which wasn't popular, so let's use this workaround instead: The finish_arch_post_lock_switch() definition is now also hidden inside of #ifdef MODULE, so we don't see anything referencing preempt_enable_no_resched() from a header file. I've built a few hundred randconfig kernels with this, and did not see any new problems. BUG=chromium:779077 TEST=Build, boot on caroline. Tested-by: Guenter Roeck <linux@roeck-us.net> Signed-off-by: Steven Rostedt <rostedt@goodmis.org> Signed-off-by: Arnd Bergmann <arnd@arndb.de> Acked-by: Russell King <rmk+kernel@arm.linux.org.uk> Cc: Alexander Shishkin <alexander.shishkin@linux.intel.com> Cc: Andy Lutomirski <luto@amacapital.net> Cc: Andy Lutomirski <luto@kernel.org> Cc: Ard Biesheuvel <ard.biesheuvel@linaro.org> Cc: Arnaldo Carvalho de Melo <acme@redhat.com> Cc: Borislav Petkov <bp@suse.de> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Linus Torvalds <torvalds@linux-foundation.org> Cc: Mel Gorman <mgorman@techsingularity.net> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Russell King - ARM Linux <linux@armlinux.org.uk> Cc: Stephane Eranian <eranian@google.com> Cc: Thomas Gleixner <tglx@linutronix.de> Cc: Vince Weaver <vincent.weaver@maine.edu> Cc: linux-arm-kernel@lists.infradead.org Fixes: f98db6013c55 ("sched/core: Add switch_mm_irqs_off() and use it in the scheduler") Link: http://lkml.kernel.org/r/1463146234-161304-1-git-send-email-arnd@arndb.de Signed-off-by: Ingo Molnar <mingo@kernel.org> (cherry picked from commit ef0491ea17f8019821c7e9c8e801184ecf17f85a) Change-Id: I2d9b447238d9b06a2adc812e3139e7d77313c34a Reviewed-on: https://chromium-review.googlesource.com/742011 Commit-Ready: Jorge Lucangeli Obes <jorgelo@chromium.org> Tested-by: Jorge Lucangeli Obes <jorgelo@chromium.org> Reviewed-by: Guenter Roeck <groeck@chromium.org> Reviewed-on: https://chromium-review.googlesource.com/764290 Reviewed-by: Jorge Lucangeli Obes <jorgelo@chromium.org> Commit-Queue: Jorge Lucangeli Obes <jorgelo@chromium.org> [modify] https://crrev.com/d4f7d9d62e4151b68a025244cbcfc7922d861caf/arch/arm/include/asm/mmu_context.h
,
Nov 11 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromiumos/third_party/kernel/+/193ef5f086fdf6644891e495511eb7ffec2d6305 commit 193ef5f086fdf6644891e495511eb7ffec2d6305 Author: Andy Lutomirski <luto@kernel.org> Date: Sat Nov 11 00:30:14 2017 UPSTREAM: sched/core: Idle_task_exit() shouldn't use switch_mm_irqs_off() idle_task_exit() can be called with IRQs on x86 on and therefore should use switch_mm(), not switch_mm_irqs_off(). This doesn't seem to cause any problems right now, but it will confuse my upcoming TLB flush changes. Nonetheless, I think it should be backported because it's trivial. There won't be any meaningful performance impact because idle_task_exit() is only used when offlining a CPU. BUG=chromium:779077 TEST=Build, boot on caroline. Signed-off-by: Andy Lutomirski <luto@kernel.org> Cc: Borislav Petkov <bp@suse.de> Cc: Linus Torvalds <torvalds@linux-foundation.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Thomas Gleixner <tglx@linutronix.de> Cc: stable@vger.kernel.org Fixes: f98db6013c55 ("sched/core: Add switch_mm_irqs_off() and use it in the scheduler") Link: http://lkml.kernel.org/r/ca3d1a9fa93a0b49f5a8ff729eda3640fb6abdf9.1497034141.git.luto@kernel.org Signed-off-by: Ingo Molnar <mingo@kernel.org> (cherry picked from commit 252d2a4117bc181b287eeddf848863788da733ae) Change-Id: Icf9b0f7b3fa6362e206ce8d6f8176d9271ba61b9 Reviewed-on: https://chromium-review.googlesource.com/742012 Commit-Ready: Jorge Lucangeli Obes <jorgelo@chromium.org> Tested-by: Jorge Lucangeli Obes <jorgelo@chromium.org> Reviewed-by: Guenter Roeck <groeck@chromium.org> Reviewed-on: https://chromium-review.googlesource.com/764291 Reviewed-by: Jorge Lucangeli Obes <jorgelo@chromium.org> Commit-Queue: Jorge Lucangeli Obes <jorgelo@chromium.org> [modify] https://crrev.com/193ef5f086fdf6644891e495511eb7ffec2d6305/kernel/sched/core.c
,
Nov 11 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromiumos/third_party/kernel/+/1d64a23e0c83ddf7bce54ea3eae219a465b9ffe1 commit 1d64a23e0c83ddf7bce54ea3eae219a465b9ffe1 Author: Sebastian Andrzej Siewior <bigeasy@linutronix.de> Date: Sat Nov 11 00:30:18 2017 UPSTREAM: x86/mm: Disable preemption during CR3 read+write There's a subtle preemption race on UP kernels: Usually current->mm (and therefore mm->pgd) stays the same during the lifetime of a task so it does not matter if a task gets preempted during the read and write of the CR3. But then, there is this scenario on x86-UP: TaskA is in do_exit() and exit_mm() sets current->mm = NULL followed by: -> mmput() -> exit_mmap() -> tlb_finish_mmu() -> tlb_flush_mmu() -> tlb_flush_mmu_tlbonly() -> tlb_flush() -> flush_tlb_mm_range() -> __flush_tlb_up() -> __flush_tlb() -> __native_flush_tlb() At this point current->mm is NULL but current->active_mm still points to the "old" mm. Let's preempt taskA _after_ native_read_cr3() by taskB. TaskB has its own mm so CR3 has changed. Now preempt back to taskA. TaskA has no ->mm set so it borrows taskB's mm and so CR3 remains unchanged. Once taskA gets active it continues where it was interrupted and that means it writes its old CR3 value back. Everything is fine because userland won't need its memory anymore. Now the fun part: Let's preempt taskA one more time and get back to taskB. This time switch_mm() won't do a thing because oldmm (->active_mm) is the same as mm (as per context_switch()). So we remain with a bad CR3 / PGD and return to userland. The next thing that happens is handle_mm_fault() with an address for the execution of its code in userland. handle_mm_fault() realizes that it has a PTE with proper rights so it returns doing nothing. But the CPU looks at the wrong PGD and insists that something is wrong and faults again. And again. And one more time This pagefault circle continues until the scheduler gets tired of it and puts another task on the CPU. It gets little difficult if the task is a RT task with a high priority. The system will either freeze or it gets fixed by the software watchdog thread which usually runs at RT-max prio. But waiting for the watchdog will increase the latency of the RT task which is no good. Fix this by disabling preemption across the critical code section. BUG=chromium:779077 TEST=Build, boot on caroline. Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de> Acked-by: Peter Zijlstra (Intel) <peterz@infradead.org> Acked-by: Rik van Riel <riel@redhat.com> Acked-by: Andy Lutomirski <luto@kernel.org> Cc: Borislav Petkov <bp@alien8.de> Cc: Borislav Petkov <bp@suse.de> Cc: Brian Gerst <brgerst@gmail.com> Cc: Denys Vlasenko <dvlasenk@redhat.com> Cc: H. Peter Anvin <hpa@zytor.com> Cc: Josh Poimboeuf <jpoimboe@redhat.com> Cc: Linus Torvalds <torvalds@linux-foundation.org> Cc: Mel Gorman <mgorman@suse.de> Cc: Peter Zijlstra <a.p.zijlstra@chello.nl> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Thomas Gleixner <tglx@linutronix.de> Cc: linux-mm@kvack.org Cc: stable@vger.kernel.org Link: http://lkml.kernel.org/r/1470404259-26290-1-git-send-email-bigeasy@linutronix.de [ Prettified the changelog. ] Signed-off-by: Ingo Molnar <mingo@kernel.org> (cherry picked from commit 5cf0791da5c162ebc14b01eb01631cfa7ed4fa6e) Change-Id: Idfbcc9265368751dcbbb5e91fc35ec2b60d9a6c4 Reviewed-on: https://chromium-review.googlesource.com/742013 Commit-Ready: Jorge Lucangeli Obes <jorgelo@chromium.org> Tested-by: Jorge Lucangeli Obes <jorgelo@chromium.org> Reviewed-by: Guenter Roeck <groeck@chromium.org> Reviewed-on: https://chromium-review.googlesource.com/764292 Reviewed-by: Jorge Lucangeli Obes <jorgelo@chromium.org> Commit-Queue: Jorge Lucangeli Obes <jorgelo@chromium.org> [modify] https://crrev.com/1d64a23e0c83ddf7bce54ea3eae219a465b9ffe1/arch/x86/include/asm/tlbflush.h
,
Nov 11 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromiumos/third_party/kernel/+/419ef8191f5611f6025847326770fc8bc326d9da commit 419ef8191f5611f6025847326770fc8bc326d9da Author: Aaron Lu <aaron.lu@intel.com> Date: Sat Nov 11 00:30:22 2017 UPSTREAM: x86/irq: Do not substract irq_tlb_count from irq_call_count Since commit: 52aec3308db8 ("x86/tlb: replace INVALIDATE_TLB_VECTOR by CALL_FUNCTION_VECTOR") the TLB remote shootdown is done through call function vector. That commit didn't take care of irq_tlb_count, which a later commit: fd0f5869724f ("x86: Distinguish TLB shootdown interrupts from other functions call interrupts") ... tried to fix. The fix assumes every increase of irq_tlb_count has a corresponding increase of irq_call_count. So the irq_call_count is always bigger than irq_tlb_count and we could substract irq_tlb_count from irq_call_count. Unfortunately this is not true for the smp_call_function_single() case. The IPI is only sent if the target CPU's call_single_queue is empty when adding a csd into it in generic_exec_single. That means if two threads are both adding flush tlb csds to the same CPU's call_single_queue, only one IPI is sent. In other words, the irq_call_count is incremented by 1 but irq_tlb_count is incremented by 2. Over time, irq_tlb_count will be bigger than irq_call_count and the substract will produce a very large irq_call_count value due to overflow. Considering that: 1) it's not worth to send more IPIs for the sake of accurate counting of irq_call_count in generic_exec_single(); 2) it's not easy to tell if the call function interrupt is for TLB shootdown in __smp_call_function_single_interrupt(). Not to exclude TLB shootdown from call function count seems to be the simplest fix and this patch just does that. This bug was found by LKP's cyclic performance regression tracking recently with the vm-scalability test suite. I have bisected to commit: 3dec0ba0be6a ("mm/rmap: share the i_mmap_rwsem") This commit didn't do anything wrong but revealed the irq_call_count problem. IIUC, the commit makes rwc->remap_one in rmap_walk_file concurrent with multiple threads. When remap_one is try_to_unmap_one(), then multiple threads could queue flush TLB to the same CPU but only one IPI will be sent. Since the commit was added in Linux v3.19, the counting problem only shows up from v3.19 onwards. BUG=chromium:779077 TEST=Build, boot on caroline. Signed-off-by: Aaron Lu <aaron.lu@intel.com> Cc: Alex Shi <alex.shi@linaro.org> Cc: Andy Lutomirski <luto@kernel.org> Cc: Borislav Petkov <bp@alien8.de> Cc: Brian Gerst <brgerst@gmail.com> Cc: Davidlohr Bueso <dave@stgolabs.net> Cc: Denys Vlasenko <dvlasenk@redhat.com> Cc: H. Peter Anvin <hpa@zytor.com> Cc: Huang Ying <ying.huang@intel.com> Cc: Josh Poimboeuf <jpoimboe@redhat.com> Cc: Linus Torvalds <torvalds@linux-foundation.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Thomas Gleixner <tglx@linutronix.de> Cc: Tomoki Sekiyama <tomoki.sekiyama.qu@hitachi.com> Link: http://lkml.kernel.org/r/20160811074430.GA18163@aaronlu.sh.intel.com Signed-off-by: Ingo Molnar <mingo@kernel.org> (cherry picked from commit 82ba4faca1bffad429f15c90c980ffd010366c25) Change-Id: Ie1344df19f91ef439f927b2be63fec8f4e35ac72 Reviewed-on: https://chromium-review.googlesource.com/742014 Commit-Ready: Jorge Lucangeli Obes <jorgelo@chromium.org> Tested-by: Jorge Lucangeli Obes <jorgelo@chromium.org> Reviewed-by: Guenter Roeck <groeck@chromium.org> Reviewed-on: https://chromium-review.googlesource.com/764293 Reviewed-by: Jorge Lucangeli Obes <jorgelo@chromium.org> Commit-Queue: Jorge Lucangeli Obes <jorgelo@chromium.org> [modify] https://crrev.com/419ef8191f5611f6025847326770fc8bc326d9da/arch/x86/kernel/irq.c [modify] https://crrev.com/419ef8191f5611f6025847326770fc8bc326d9da/arch/x86/include/asm/hardirq.h
,
Nov 11 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromiumos/third_party/kernel/+/573af891ed1880c44fe98385469f2400820ef040 commit 573af891ed1880c44fe98385469f2400820ef040 Author: Andy Lutomirski <luto@kernel.org> Date: Sat Nov 11 00:30:25 2017 UPSTREAM: x86/vm86/32: Switch to flush_tlb_mm_range() in mark_screen_rdonly() mark_screen_rdonly() is the last remaining caller of flush_tlb(). flush_tlb_mm_range() is potentially faster and isn't obsolete. Compile-tested only because I don't know whether software that uses this mechanism even exists. BUG=chromium:779077 TEST=Build, boot on caroline. Signed-off-by: Andy Lutomirski <luto@kernel.org> Cc: Andrew Morton <akpm@linux-foundation.org> Cc: Borislav Petkov <bp@alien8.de> Cc: Brian Gerst <brgerst@gmail.com> Cc: Dave Hansen <dave.hansen@intel.com> Cc: Denys Vlasenko <dvlasenk@redhat.com> Cc: H. Peter Anvin <hpa@zytor.com> Cc: Josh Poimboeuf <jpoimboe@redhat.com> Cc: Linus Torvalds <torvalds@linux-foundation.org> Cc: Michal Hocko <mhocko@suse.com> Cc: Nadav Amit <namit@vmware.com> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Rik van Riel <riel@redhat.com> Cc: Sasha Levin <sasha.levin@oracle.com> Cc: Thomas Gleixner <tglx@linutronix.de> Link: http://lkml.kernel.org/r/791a644076fc3577ba7f7b7cafd643cc089baa7d.1492844372.git.luto@kernel.org Signed-off-by: Ingo Molnar <mingo@kernel.org> (cherry picked from commit 9ccee2373f0658f234727700e619df097ba57023) Change-Id: I4d954a69c46567e6b36b2f239a34687121f11d4a Reviewed-on: https://chromium-review.googlesource.com/742015 Commit-Ready: Jorge Lucangeli Obes <jorgelo@chromium.org> Tested-by: Jorge Lucangeli Obes <jorgelo@chromium.org> Reviewed-by: Guenter Roeck <groeck@chromium.org> Reviewed-on: https://chromium-review.googlesource.com/764294 Reviewed-by: Jorge Lucangeli Obes <jorgelo@chromium.org> Commit-Queue: Jorge Lucangeli Obes <jorgelo@chromium.org> [modify] https://crrev.com/573af891ed1880c44fe98385469f2400820ef040/arch/x86/kernel/vm86_32.c
,
Nov 11 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromiumos/third_party/kernel/+/705e21fb209109ba9dae70ef48c23dabcaafb795 commit 705e21fb209109ba9dae70ef48c23dabcaafb795 Author: Andy Lutomirski <luto@kernel.org> Date: Sat Nov 11 00:30:29 2017 UPSTREAM: x86/mm: Remove flush_tlb() and flush_tlb_current_task() I was trying to figure out what how flush_tlb_current_task() would possibly work correctly if current->mm != current->active_mm, but I realized I could spare myself the effort: it has no callers except the unused flush_tlb() macro. BUG=chromium:779077 TEST=Build, boot on caroline. Signed-off-by: Andy Lutomirski <luto@kernel.org> Cc: Andrew Morton <akpm@linux-foundation.org> Cc: Borislav Petkov <bp@alien8.de> Cc: Brian Gerst <brgerst@gmail.com> Cc: Dave Hansen <dave.hansen@intel.com> Cc: Denys Vlasenko <dvlasenk@redhat.com> Cc: H. Peter Anvin <hpa@zytor.com> Cc: Josh Poimboeuf <jpoimboe@redhat.com> Cc: Linus Torvalds <torvalds@linux-foundation.org> Cc: Michal Hocko <mhocko@suse.com> Cc: Nadav Amit <namit@vmware.com> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Rik van Riel <riel@redhat.com> Cc: Thomas Gleixner <tglx@linutronix.de> Link: http://lkml.kernel.org/r/e52d64c11690f85e9f1d69d7b48cc2269cd2e94b.1492844372.git.luto@kernel.org Signed-off-by: Ingo Molnar <mingo@kernel.org> (cherry picked from commit 29961b59a51f8c6838a26a45e871a7ed6771809b) Change-Id: I96801f3da7943c0a3d949307355289cf1470a5d5 Reviewed-on: https://chromium-review.googlesource.com/742016 Commit-Ready: Jorge Lucangeli Obes <jorgelo@chromium.org> Tested-by: Jorge Lucangeli Obes <jorgelo@chromium.org> Reviewed-by: Guenter Roeck <groeck@chromium.org> Reviewed-on: https://chromium-review.googlesource.com/764295 Reviewed-by: Jorge Lucangeli Obes <jorgelo@chromium.org> Commit-Queue: Jorge Lucangeli Obes <jorgelo@chromium.org> [modify] https://crrev.com/705e21fb209109ba9dae70ef48c23dabcaafb795/arch/x86/include/asm/tlbflush.h [modify] https://crrev.com/705e21fb209109ba9dae70ef48c23dabcaafb795/arch/x86/mm/tlb.c
,
Nov 11 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromiumos/third_party/kernel/+/4413b9acb99fb84ca0680ae637b2dc831d298409 commit 4413b9acb99fb84ca0680ae637b2dc831d298409 Author: Andy Lutomirski <luto@kernel.org> Date: Sat Nov 11 00:30:33 2017 UPSTREAM: x86/mm: Make flush_tlb_mm_range() more predictable I'm about to rewrite the function almost completely, but first I want to get a functional change out of the way. Currently, if flush_tlb_mm_range() does not flush the local TLB at all, it will never do individual page flushes on remote CPUs. This seems to be an accident, and preserving it will be awkward. Let's change it first so that any regressions in the rewrite will be easier to bisect and so that the rewrite can attempt to change no visible behavior at all. The fix is simple: we can simply avoid short-circuiting the calculation of base_pages_to_flush. As a side effect, this also eliminates a potential corner case: if tlb_single_page_flush_ceiling == TLB_FLUSH_ALL, flush_tlb_mm_range() could have ended up flushing the entire address space one page at a time. BUG=chromium:779077 TEST=Build, boot on caroline. Signed-off-by: Andy Lutomirski <luto@kernel.org> Acked-by: Dave Hansen <dave.hansen@intel.com> Cc: Andrew Morton <akpm@linux-foundation.org> Cc: Borislav Petkov <bp@alien8.de> Cc: Brian Gerst <brgerst@gmail.com> Cc: Denys Vlasenko <dvlasenk@redhat.com> Cc: H. Peter Anvin <hpa@zytor.com> Cc: Josh Poimboeuf <jpoimboe@redhat.com> Cc: Linus Torvalds <torvalds@linux-foundation.org> Cc: Michal Hocko <mhocko@suse.com> Cc: Nadav Amit <namit@vmware.com> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Rik van Riel <riel@redhat.com> Cc: Thomas Gleixner <tglx@linutronix.de> Link: http://lkml.kernel.org/r/4b29b771d9975aad7154c314534fec235618175a.1492844372.git.luto@kernel.org Signed-off-by: Ingo Molnar <mingo@kernel.org> (cherry picked from commit ce27374fabf553153c3f53efcaa9bfab9216bd8c) Change-Id: Ib0c14085117e89bc3b3465c095ef0a86fe0641ce Reviewed-on: https://chromium-review.googlesource.com/742017 Commit-Ready: Jorge Lucangeli Obes <jorgelo@chromium.org> Tested-by: Jorge Lucangeli Obes <jorgelo@chromium.org> Reviewed-by: Guenter Roeck <groeck@chromium.org> Reviewed-on: https://chromium-review.googlesource.com/764296 Reviewed-by: Jorge Lucangeli Obes <jorgelo@chromium.org> Commit-Queue: Jorge Lucangeli Obes <jorgelo@chromium.org> [modify] https://crrev.com/4413b9acb99fb84ca0680ae637b2dc831d298409/arch/x86/mm/tlb.c
,
Nov 11 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromiumos/third_party/kernel/+/33bae11e0b8961342c884b82fde44f7d79da9917 commit 33bae11e0b8961342c884b82fde44f7d79da9917 Author: Andy Lutomirski <luto@kernel.org> Date: Sat Nov 11 00:30:37 2017 UPSTREAM: x86/mm: Fix flush_tlb_page() on Xen flush_tlb_page() passes a bogus range to flush_tlb_others() and expects the latter to fix it up. native_flush_tlb_others() has the fixup but Xen's version doesn't. Move the fixup to flush_tlb_others(). AFAICS the only real effect is that, without this fix, Xen would flush everything instead of just the one page on remote vCPUs in when flush_tlb_page() was called. BUG=chromium:779077 TEST=Build, boot on caroline. Signed-off-by: Andy Lutomirski <luto@kernel.org> Reviewed-by: Boris Ostrovsky <boris.ostrovsky@oracle.com> Cc: Andrew Morton <akpm@linux-foundation.org> Cc: Borislav Petkov <bp@alien8.de> Cc: Brian Gerst <brgerst@gmail.com> Cc: Dave Hansen <dave.hansen@intel.com> Cc: Denys Vlasenko <dvlasenk@redhat.com> Cc: H. Peter Anvin <hpa@zytor.com> Cc: Josh Poimboeuf <jpoimboe@redhat.com> Cc: Juergen Gross <jgross@suse.com> Cc: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com> Cc: Linus Torvalds <torvalds@linux-foundation.org> Cc: Michal Hocko <mhocko@suse.com> Cc: Nadav Amit <namit@vmware.com> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Rik van Riel <riel@redhat.com> Cc: Thomas Gleixner <tglx@linutronix.de> Fixes: e7b52ffd45a6 ("x86/flush_tlb: try flush_tlb_single one by one in flush_tlb_range") Link: http://lkml.kernel.org/r/10ed0e4dfea64daef10b87fb85df1746999b4dba.1492844372.git.luto@kernel.org Signed-off-by: Ingo Molnar <mingo@kernel.org> (cherry picked from commit dbd68d8e84c606673ebbcf15862f8c155fa92326) Change-Id: Ie88fe5eebb01cd6ac7af138b5f2af0c47307d9df Reviewed-on: https://chromium-review.googlesource.com/742018 Commit-Ready: Jorge Lucangeli Obes <jorgelo@chromium.org> Tested-by: Jorge Lucangeli Obes <jorgelo@chromium.org> Reviewed-by: Guenter Roeck <groeck@chromium.org> Reviewed-on: https://chromium-review.googlesource.com/764297 Reviewed-by: Jorge Lucangeli Obes <jorgelo@chromium.org> Commit-Queue: Jorge Lucangeli Obes <jorgelo@chromium.org> [modify] https://crrev.com/33bae11e0b8961342c884b82fde44f7d79da9917/arch/x86/mm/tlb.c
,
Nov 11 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromiumos/third_party/kernel/+/74789e663297291cebf1f852d98885229ae4f8a6 commit 74789e663297291cebf1f852d98885229ae4f8a6 Author: Andy Lutomirski <luto@kernel.org> Date: Sat Nov 11 00:30:41 2017 UPSTREAM: x86/mm: Reimplement flush_tlb_page() using flush_tlb_mm_range() flush_tlb_page() was very similar to flush_tlb_mm_range() except that it had a couple of issues: - It was missing an smp_mb() in the case where current->active_mm != mm. (This is a longstanding bug reported by Nadav Amit) - It was missing tracepoints and vm counter updates. The only reason that I can see for keeping it at as a separate function is that it could avoid a few branches that flush_tlb_mm_range() needs to decide to flush just one page. This hardly seems worthwhile. If we decide we want to get rid of those branches again, a better way would be to introduce an __flush_tlb_mm_range() helper and make both flush_tlb_page() and flush_tlb_mm_range() use it. BUG=chromium:779077 TEST=Build, boot on caroline. Signed-off-by: Andy Lutomirski <luto@kernel.org> Acked-by: Kees Cook <keescook@chromium.org> Cc: Andrew Morton <akpm@linux-foundation.org> Cc: Borislav Petkov <bpetkov@suse.de> Cc: Dave Hansen <dave.hansen@intel.com> Cc: Linus Torvalds <torvalds@linux-foundation.org> Cc: Mel Gorman <mgorman@suse.de> Cc: Michal Hocko <mhocko@suse.com> Cc: Nadav Amit <nadav.amit@gmail.com> Cc: Nadav Amit <namit@vmware.com> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Rik van Riel <riel@redhat.com> Cc: Thomas Gleixner <tglx@linutronix.de> Cc: linux-mm@kvack.org Link: http://lkml.kernel.org/r/3cc3847cf888d8907577569b8bac3f01992ef8f9.1495492063.git.luto@kernel.org Signed-off-by: Ingo Molnar <mingo@kernel.org> (cherry picked from commit ca6c99c0794875c6d1db6e22f246699691ab7e6b) Change-Id: If7588124f68be57bf34c3292f457ff7a92bbd581 Reviewed-on: https://chromium-review.googlesource.com/742019 Commit-Ready: Jorge Lucangeli Obes <jorgelo@chromium.org> Tested-by: Jorge Lucangeli Obes <jorgelo@chromium.org> Reviewed-by: Guenter Roeck <groeck@chromium.org> Reviewed-on: https://chromium-review.googlesource.com/764298 Reviewed-by: Jorge Lucangeli Obes <jorgelo@chromium.org> Commit-Queue: Jorge Lucangeli Obes <jorgelo@chromium.org> [modify] https://crrev.com/74789e663297291cebf1f852d98885229ae4f8a6/arch/x86/include/asm/tlbflush.h [modify] https://crrev.com/74789e663297291cebf1f852d98885229ae4f8a6/arch/x86/mm/tlb.c
,
Nov 11 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromiumos/third_party/kernel/+/513cee479d62ad087aaf999bc363a384a902f1ac commit 513cee479d62ad087aaf999bc363a384a902f1ac Author: Andy Lutomirski <luto@kernel.org> Date: Sat Nov 11 00:30:44 2017 BACKPORT: x86/mm: Remove the UP asm/tlbflush.h code, always use the (formerly) SMP code The UP asm/tlbflush.h generates somewhat nicer code than the SMP version. Aside from that, it's fallen quite a bit behind the SMP code: - flush_tlb_mm_range() didn't flush individual pages if the range was small. - The lazy TLB code was much weaker. This usually wouldn't matter, but, if a kernel thread flushed its lazy "active_mm" more than once (due to reclaim or similar), it wouldn't be unlazied and would instead pointlessly flush repeatedly. - Tracepoints were missing. Aside from that, simply having the UP code around was a maintanence burden, since it means that any change to the TLB flush code had to make sure not to break it. Simplify everything by deleting the UP code. BUG=chromium:779077 TEST=Build, boot on caroline. Signed-off-by: Andy Lutomirski <luto@kernel.org> Cc: Andrew Morton <akpm@linux-foundation.org> Cc: Arjan van de Ven <arjan@linux.intel.com> Cc: Borislav Petkov <bpetkov@suse.de> Cc: Dave Hansen <dave.hansen@intel.com> Cc: Linus Torvalds <torvalds@linux-foundation.org> Cc: Mel Gorman <mgorman@suse.de> Cc: Michal Hocko <mhocko@suse.com> Cc: Nadav Amit <nadav.amit@gmail.com> Cc: Nadav Amit <namit@vmware.com> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Rik van Riel <riel@redhat.com> Cc: Thomas Gleixner <tglx@linutronix.de> Cc: linux-mm@kvack.org Signed-off-by: Ingo Molnar <mingo@kernel.org> (cherry picked from commit ce4a4e565f5264909a18c733b864c3f74467f69e) Conflicts: arch/x86/Kconfig arch/x86/include/asm/tlbbatch.h arch/x86/include/asm/tlbflush.h arch/x86/mm/tlb.c Change-Id: Ic698d21af4235540c79ca5c68d5e6d93d3242a8b Reviewed-on: https://chromium-review.googlesource.com/742020 Commit-Ready: Jorge Lucangeli Obes <jorgelo@chromium.org> Tested-by: Jorge Lucangeli Obes <jorgelo@chromium.org> Reviewed-by: Guenter Roeck <groeck@chromium.org> Reviewed-on: https://chromium-review.googlesource.com/764299 Reviewed-by: Jorge Lucangeli Obes <jorgelo@chromium.org> Commit-Queue: Jorge Lucangeli Obes <jorgelo@chromium.org> [modify] https://crrev.com/513cee479d62ad087aaf999bc363a384a902f1ac/arch/x86/include/asm/mmu_context.h [modify] https://crrev.com/513cee479d62ad087aaf999bc363a384a902f1ac/arch/x86/include/asm/hardirq.h [modify] https://crrev.com/513cee479d62ad087aaf999bc363a384a902f1ac/arch/x86/mm/init.c [modify] https://crrev.com/513cee479d62ad087aaf999bc363a384a902f1ac/arch/x86/include/asm/tlbflush.h [modify] https://crrev.com/513cee479d62ad087aaf999bc363a384a902f1ac/arch/x86/mm/tlb.c [modify] https://crrev.com/513cee479d62ad087aaf999bc363a384a902f1ac/arch/x86/include/asm/mmu.h
,
Nov 11 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromiumos/third_party/kernel/+/4be6ffe1122d6f2ed73ff491ddf56d7fcab6b2bd commit 4be6ffe1122d6f2ed73ff491ddf56d7fcab6b2bd Author: Andy Lutomirski <luto@kernel.org> Date: Sat Nov 11 00:30:48 2017 UPSTREAM: x86/mm: Disable PCID on 32-bit kernels 32-bit kernels on new hardware will see PCID in CPUID, but PCID can only be used in 64-bit mode. Rather than making all PCID code conditional, just disable the feature on 32-bit builds. BUG=chromium:779077 TEST=Build, boot on caroline. Signed-off-by: Andy Lutomirski <luto@kernel.org> Reviewed-by: Nadav Amit <nadav.amit@gmail.com> Reviewed-by: Borislav Petkov <bp@suse.de> Reviewed-by: Thomas Gleixner <tglx@linutronix.de> Cc: Andrew Morton <akpm@linux-foundation.org> Cc: Arjan van de Ven <arjan@linux.intel.com> Cc: Borislav Petkov <bp@alien8.de> Cc: Dave Hansen <dave.hansen@intel.com> Cc: Linus Torvalds <torvalds@linux-foundation.org> Cc: Mel Gorman <mgorman@suse.de> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Rik van Riel <riel@redhat.com> Cc: linux-mm@kvack.org Link: http://lkml.kernel.org/r/2e391769192a4d31b808410c383c6bf0734bc6ea.1498751203.git.luto@kernel.org Signed-off-by: Ingo Molnar <mingo@kernel.org> (cherry picked from commit cba4671af7550e008f7a7835f06df0763825bf3e) Change-Id: Iad0f97cfa22b4c9924cfea5e7678a36e8e81a65e Reviewed-on: https://chromium-review.googlesource.com/742021 Commit-Ready: Jorge Lucangeli Obes <jorgelo@chromium.org> Tested-by: Jorge Lucangeli Obes <jorgelo@chromium.org> Reviewed-by: Guenter Roeck <groeck@chromium.org> Reviewed-on: https://chromium-review.googlesource.com/764300 Reviewed-by: Jorge Lucangeli Obes <jorgelo@chromium.org> Commit-Queue: Jorge Lucangeli Obes <jorgelo@chromium.org> [modify] https://crrev.com/4be6ffe1122d6f2ed73ff491ddf56d7fcab6b2bd/arch/x86/include/asm/disabled-features.h [modify] https://crrev.com/4be6ffe1122d6f2ed73ff491ddf56d7fcab6b2bd/arch/x86/kernel/cpu/bugs.c
,
Nov 11 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromiumos/third_party/kernel/+/156766c960ed997e46ebdb7ba2ed8c4dbeb23a7c commit 156766c960ed997e46ebdb7ba2ed8c4dbeb23a7c Author: Andy Lutomirski <luto@kernel.org> Date: Sat Nov 11 00:30:54 2017 BACKPORT: x86/mm: Add the 'nopcid' boot option to turn off PCID The parameter is only present on x86_64 systems to save a few bytes, as PCID is always disabled on x86_32. BUG=chromium:779077 TEST=Build, boot on caroline. Signed-off-by: Andy Lutomirski <luto@kernel.org> Reviewed-by: Nadav Amit <nadav.amit@gmail.com> Reviewed-by: Borislav Petkov <bp@suse.de> Reviewed-by: Thomas Gleixner <tglx@linutronix.de> Cc: Andrew Morton <akpm@linux-foundation.org> Cc: Arjan van de Ven <arjan@linux.intel.com> Cc: Borislav Petkov <bp@alien8.de> Cc: Dave Hansen <dave.hansen@intel.com> Cc: Linus Torvalds <torvalds@linux-foundation.org> Cc: Mel Gorman <mgorman@suse.de> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Rik van Riel <riel@redhat.com> Cc: linux-mm@kvack.org Link: http://lkml.kernel.org/r/8bbb2e65bcd249a5f18bfb8128b4689f08ac2b60.1498751203.git.luto@kernel.org Signed-off-by: Ingo Molnar <mingo@kernel.org> (cherry picked from commit 0790c9aad84901ca1bdc14746175549c8b5da215) Conflicts: Documentation/admin-guide/kernel-parameters.txt (not in this tree) Documentation/kernel-parameters.txt (patched instead of that) Change-Id: I8e87aa6680b2586729ef1ed6f83626b90330dfed Reviewed-on: https://chromium-review.googlesource.com/742022 Commit-Ready: Jorge Lucangeli Obes <jorgelo@chromium.org> Tested-by: Jorge Lucangeli Obes <jorgelo@chromium.org> Reviewed-by: Guenter Roeck <groeck@chromium.org> Reviewed-on: https://chromium-review.googlesource.com/764301 Reviewed-by: Jorge Lucangeli Obes <jorgelo@chromium.org> Commit-Queue: Jorge Lucangeli Obes <jorgelo@chromium.org> [modify] https://crrev.com/156766c960ed997e46ebdb7ba2ed8c4dbeb23a7c/Documentation/kernel-parameters.txt [modify] https://crrev.com/156766c960ed997e46ebdb7ba2ed8c4dbeb23a7c/arch/x86/kernel/cpu/common.c
,
Nov 11 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromiumos/third_party/kernel/+/90172da681f92d1afeddbbf66bfe840a1e6a78c2 commit 90172da681f92d1afeddbbf66bfe840a1e6a78c2 Author: Andy Lutomirski <luto@kernel.org> Date: Sat Nov 11 00:30:59 2017 BACKPORT: x86/mm: Enable CR4.PCIDE on supported systems We can use PCID if the CPU has PCID and PGE and we're not on Xen. By itself, this has no effect. A followup patch will start using PCID. BUG=chromium:779077 TEST=Build, boot on caroline. Signed-off-by: Andy Lutomirski <luto@kernel.org> Reviewed-by: Nadav Amit <nadav.amit@gmail.com> Reviewed-by: Boris Ostrovsky <boris.ostrovsky@oracle.com> Reviewed-by: Thomas Gleixner <tglx@linutronix.de> Cc: Andrew Morton <akpm@linux-foundation.org> Cc: Arjan van de Ven <arjan@linux.intel.com> Cc: Borislav Petkov <bp@alien8.de> Cc: Dave Hansen <dave.hansen@intel.com> Cc: Juergen Gross <jgross@suse.com> Cc: Linus Torvalds <torvalds@linux-foundation.org> Cc: Mel Gorman <mgorman@suse.de> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Rik van Riel <riel@redhat.com> Cc: linux-mm@kvack.org Link: http://lkml.kernel.org/r/6327ecd907b32f79d5aa0d466f04503bbec5df88.1498751203.git.luto@kernel.org Signed-off-by: Ingo Molnar <mingo@kernel.org> (cherry picked from commit 660da7c9228f685b2ebe664f9fd69aaddcc420b5) Conflicts: arch/x86/xen/enlighten_pv.c (not in this tree) arch/x86/xen/enlighten.c (patched instead of that) Change-Id: I9c3316f75c91c8ab831ce3fe55a283e74189f3c2 Reviewed-on: https://chromium-review.googlesource.com/742023 Commit-Ready: Jorge Lucangeli Obes <jorgelo@chromium.org> Tested-by: Jorge Lucangeli Obes <jorgelo@chromium.org> Reviewed-by: Guenter Roeck <groeck@chromium.org> Reviewed-on: https://chromium-review.googlesource.com/764302 Reviewed-by: Jorge Lucangeli Obes <jorgelo@chromium.org> Commit-Queue: Jorge Lucangeli Obes <jorgelo@chromium.org> [modify] https://crrev.com/90172da681f92d1afeddbbf66bfe840a1e6a78c2/arch/x86/include/asm/tlbflush.h [modify] https://crrev.com/90172da681f92d1afeddbbf66bfe840a1e6a78c2/arch/x86/xen/enlighten.c [modify] https://crrev.com/90172da681f92d1afeddbbf66bfe840a1e6a78c2/arch/x86/kernel/cpu/common.c
,
Nov 13 2017
This issue has been approved for a merge. Please merge the fix to any appropriate branches as soon as possible! If all merges have been completed, please remove any remaining Merge-Approved labels from this issue. Thanks for your time! To disable nags, add the Disable-Nags label. For more details visit https://www.chromium.org/issue-tracking/autotriage - Your friendly Sheriffbot
,
Nov 13 2017
This 63 merge is done.
,
Nov 13 2017
,
Nov 16 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromiumos/third_party/kernel/+/e0c6caf3f7806f149fa0614b7f337bdce00b24ae commit e0c6caf3f7806f149fa0614b7f337bdce00b24ae Author: Andy Lutomirski <luto@kernel.org> Date: Thu Nov 16 19:38:33 2017 UPSTREAM: x86/mm: Add INVPCID helpers This adds helpers for each of the four currently-specified INVPCID modes. BUG=chromium:779077 TEST=Build, boot on soraka. Signed-off-by: Andy Lutomirski <luto@kernel.org> Reviewed-by: Borislav Petkov <bp@suse.de> Cc: Andrew Morton <akpm@linux-foundation.org> Cc: Andrey Ryabinin <aryabinin@virtuozzo.com> Cc: Andy Lutomirski <luto@amacapital.net> Cc: Borislav Petkov <bp@alien8.de> Cc: Brian Gerst <brgerst@gmail.com> Cc: Dave Hansen <dave.hansen@linux.intel.com> Cc: Denys Vlasenko <dvlasenk@redhat.com> Cc: H. Peter Anvin <hpa@zytor.com> Cc: Linus Torvalds <torvalds@linux-foundation.org> Cc: Luis R. Rodriguez <mcgrof@suse.com> Cc: Oleg Nesterov <oleg@redhat.com> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Thomas Gleixner <tglx@linutronix.de> Cc: Toshi Kani <toshi.kani@hp.com> Cc: linux-mm@kvack.org Link: http://lkml.kernel.org/r/8a62b23ad686888cee01da134c91409e22064db9.1454096309.git.luto@kernel.org Signed-off-by: Ingo Molnar <mingo@kernel.org> (cherry picked from commit 060a402a1ddb551455ee410de2eadd3349f2801b) Change-Id: I7a345e3cb9cd8264e25a7a438f8205bbe599c8a1 Reviewed-on: https://chromium-review.googlesource.com/741336 Commit-Ready: Jorge Lucangeli Obes <jorgelo@chromium.org> Tested-by: Jorge Lucangeli Obes <jorgelo@chromium.org> Reviewed-by: Guenter Roeck <groeck@chromium.org> (cherry picked from commit c8ba9445830fb247f20e1bc653ee15c567eb5980) Signed-off-by: Daniel Wang <wonderfly@google.com> Reviewed-on: https://chromium-review.googlesource.com/764523 Reviewed-by: Bernie Thompson <bhthompson@chromium.org> [modify] https://crrev.com/e0c6caf3f7806f149fa0614b7f337bdce00b24ae/arch/x86/include/asm/tlbflush.h
,
Nov 16 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromiumos/third_party/kernel/+/94babdd60d50e72c1a499b8da219671f11bc8935 commit 94babdd60d50e72c1a499b8da219671f11bc8935 Author: Borislav Petkov <bp@suse.de> Date: Thu Nov 16 19:38:40 2017 UPSTREAM: x86/mm: Fix INVPCID asm constraint So we want to specify the dependency on both @pcid and @addr so that the compiler doesn't reorder accesses to them *before* the TLB flush. But for that to work, we need to express this properly in the inline asm and deref the whole desc array, not the pointer to it. See clwb() for an example. This fixes the build error on 32-bit: arch/x86/include/asm/tlbflush.h: In function __invpcid: arch/x86/include/asm/tlbflush.h:26:18: error: memory input 0 is not directly addressable which gcc4.7 caught but 5.x didn't. Which is strange. :-\ BUG=chromium:779077 TEST=Build, boot on soraka. Signed-off-by: Borislav Petkov <bp@suse.de> Cc: Andrew Morton <akpm@linux-foundation.org> Cc: Andrey Ryabinin <aryabinin@virtuozzo.com> Cc: Andy Lutomirski <luto@amacapital.net> Cc: Borislav Petkov <bp@alien8.de> Cc: Brian Gerst <brgerst@gmail.com> Cc: Dave Hansen <dave.hansen@linux.intel.com> Cc: Denys Vlasenko <dvlasenk@redhat.com> Cc: H. Peter Anvin <hpa@zytor.com> Cc: Linus Torvalds <torvalds@linux-foundation.org> Cc: Luis R. Rodriguez <mcgrof@suse.com> Cc: Michael Matz <matz@suse.de> Cc: Oleg Nesterov <oleg@redhat.com> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Thomas Gleixner <tglx@linutronix.de> Cc: Toshi Kani <toshi.kani@hp.com> Cc: linux-mm@kvack.org Signed-off-by: Ingo Molnar <mingo@kernel.org> (cherry picked from commit e2c7698cd61f11d4077fdb28148b2d31b82ac848) Change-Id: I0f63f22403709f3aacaa9f3406ba67260c31acef Reviewed-on: https://chromium-review.googlesource.com/741337 Commit-Ready: Jorge Lucangeli Obes <jorgelo@chromium.org> Tested-by: Jorge Lucangeli Obes <jorgelo@chromium.org> Reviewed-by: Guenter Roeck <groeck@chromium.org> (cherry picked from commit 46beb28b260f32301d03edd8d94f71af5de56dc1) Signed-off-by: Daniel Wang <wonderfly@google.com> Reviewed-on: https://chromium-review.googlesource.com/764524 Reviewed-by: Bernie Thompson <bhthompson@chromium.org> [modify] https://crrev.com/94babdd60d50e72c1a499b8da219671f11bc8935/arch/x86/include/asm/tlbflush.h
,
Nov 16 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromiumos/third_party/kernel/+/d9855935faa7812a6f1d53dd565e7f90b36d6668 commit d9855935faa7812a6f1d53dd565e7f90b36d6668 Author: Andy Lutomirski <luto@kernel.org> Date: Thu Nov 16 19:38:45 2017 UPSTREAM: x86/mm: Add a 'noinvpcid' boot option to turn off INVPCID This adds a chicken bit to turn off INVPCID in case something goes wrong. It's an early_param() because we do TLB flushes before we parse __setup() parameters. BUG=chromium:779077 TEST=Build, boot on soraka. Signed-off-by: Andy Lutomirski <luto@kernel.org> Reviewed-by: Borislav Petkov <bp@suse.de> Cc: Andrew Morton <akpm@linux-foundation.org> Cc: Andrey Ryabinin <aryabinin@virtuozzo.com> Cc: Andy Lutomirski <luto@amacapital.net> Cc: Borislav Petkov <bp@alien8.de> Cc: Brian Gerst <brgerst@gmail.com> Cc: Dave Hansen <dave.hansen@linux.intel.com> Cc: Denys Vlasenko <dvlasenk@redhat.com> Cc: H. Peter Anvin <hpa@zytor.com> Cc: Linus Torvalds <torvalds@linux-foundation.org> Cc: Luis R. Rodriguez <mcgrof@suse.com> Cc: Oleg Nesterov <oleg@redhat.com> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Thomas Gleixner <tglx@linutronix.de> Cc: Toshi Kani <toshi.kani@hp.com> Cc: linux-mm@kvack.org Link: http://lkml.kernel.org/r/f586317ed1bc2b87aee652267e515b90051af385.1454096309.git.luto@kernel.org Signed-off-by: Ingo Molnar <mingo@kernel.org> (cherry picked from commit d12a72b844a49d4162f24cefdab30bed3f86730e) Change-Id: Iacab182f0c700c43a2f12cf14a0090751665c626 Reviewed-on: https://chromium-review.googlesource.com/741338 Commit-Ready: Jorge Lucangeli Obes <jorgelo@chromium.org> Tested-by: Jorge Lucangeli Obes <jorgelo@chromium.org> Reviewed-by: Guenter Roeck <groeck@chromium.org> (cherry picked from commit 890bf6279732cba34c9c31e0a547e5b0c2862a4a) Signed-off-by: Daniel Wang <wonderfly@google.com> Reviewed-on: https://chromium-review.googlesource.com/764525 Reviewed-by: Bernie Thompson <bhthompson@chromium.org> [modify] https://crrev.com/d9855935faa7812a6f1d53dd565e7f90b36d6668/Documentation/kernel-parameters.txt [modify] https://crrev.com/d9855935faa7812a6f1d53dd565e7f90b36d6668/arch/x86/kernel/cpu/common.c
,
Nov 16 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromiumos/third_party/kernel/+/be338dc783feb3c619686aa74cab8850c130ea59 commit be338dc783feb3c619686aa74cab8850c130ea59 Author: Andy Lutomirski <luto@kernel.org> Date: Thu Nov 16 19:38:49 2017 UPSTREAM: x86/mm: If INVPCID is available, use it to flush global mappings On my Skylake laptop, INVPCID function 2 (flush absolutely everything) takes about 376ns, whereas saving flags, twiddling CR4.PGE to flush global mappings, and restoring flags takes about 539ns. BUG=chromium:779077 TEST=Build, boot on soraka. Signed-off-by: Andy Lutomirski <luto@kernel.org> Reviewed-by: Borislav Petkov <bp@suse.de> Cc: Andrew Morton <akpm@linux-foundation.org> Cc: Andrey Ryabinin <aryabinin@virtuozzo.com> Cc: Andy Lutomirski <luto@amacapital.net> Cc: Borislav Petkov <bp@alien8.de> Cc: Brian Gerst <brgerst@gmail.com> Cc: Dave Hansen <dave.hansen@linux.intel.com> Cc: Denys Vlasenko <dvlasenk@redhat.com> Cc: H. Peter Anvin <hpa@zytor.com> Cc: Linus Torvalds <torvalds@linux-foundation.org> Cc: Luis R. Rodriguez <mcgrof@suse.com> Cc: Oleg Nesterov <oleg@redhat.com> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Thomas Gleixner <tglx@linutronix.de> Cc: Toshi Kani <toshi.kani@hp.com> Cc: linux-mm@kvack.org Link: http://lkml.kernel.org/r/ed0ef62581c0ea9c99b9bf6df726015e96d44743.1454096309.git.luto@kernel.org Signed-off-by: Ingo Molnar <mingo@kernel.org> (cherry picked from commit d8bced79af1db6734f66b42064cc773cada2ce99) Change-Id: I0ff9813f164963ae89520d4c995bd937b045863f Reviewed-on: https://chromium-review.googlesource.com/741339 Commit-Ready: Jorge Lucangeli Obes <jorgelo@chromium.org> Tested-by: Jorge Lucangeli Obes <jorgelo@chromium.org> Reviewed-by: Guenter Roeck <groeck@chromium.org> (cherry picked from commit 5a74c34a7b20d224874d9956c434b90563341f93) Signed-off-by: Daniel Wang <wonderfly@google.com> Reviewed-on: https://chromium-review.googlesource.com/764526 Reviewed-by: Bernie Thompson <bhthompson@chromium.org> [modify] https://crrev.com/be338dc783feb3c619686aa74cab8850c130ea59/arch/x86/include/asm/tlbflush.h
,
Nov 16 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromiumos/third_party/kernel/+/45f723e3cefd4f6b1bd5d40d84ff41a2063a9e71 commit 45f723e3cefd4f6b1bd5d40d84ff41a2063a9e71 Author: Nadav Amit <namit@vmware.com> Date: Thu Nov 16 19:38:53 2017 UPSTREAM: mm/rmap: batched invalidations should use existing api The recently introduced batched invalidations mechanism uses its own mechanism for shootdown. However, it does wrong accounting of interrupts (e.g., inc_irq_stat is called for local invalidations), trace-points (e.g., TLB_REMOTE_SHOOTDOWN for local invalidations) and may break some platforms as it bypasses the invalidation mechanisms of Xen and SGI UV. This patch reuses the existing TLB flushing mechnaisms instead. We use NULL as mm to indicate a global invalidation is required. BUG=chromium:779077 TEST=Build, boot on soraka. Fixes 72b252aed506b8 ("mm: send one IPI per CPU to TLB flush all entries after unmapping pages") Signed-off-by: Nadav Amit <namit@vmware.com> Cc: Mel Gorman <mgorman@suse.de> Cc: Rik van Riel <riel@redhat.com> Cc: Dave Hansen <dave.hansen@intel.com> Cc: Ingo Molnar <mingo@kernel.org> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org> (cherry picked from commit 858eaaa711700ce4595e039441e239e56d7b9514) Change-Id: Ia30c987d370400879c8afc80276cabe1a06e9c1e Reviewed-on: https://chromium-review.googlesource.com/741340 Commit-Ready: Jorge Lucangeli Obes <jorgelo@chromium.org> Tested-by: Jorge Lucangeli Obes <jorgelo@chromium.org> Reviewed-by: Guenter Roeck <groeck@chromium.org> (cherry picked from commit 86a09af748d06812d9759225384a1649733c2afc) Signed-off-by: Daniel Wang <wonderfly@google.com> Reviewed-on: https://chromium-review.googlesource.com/764927 Reviewed-by: Bernie Thompson <bhthompson@chromium.org> [modify] https://crrev.com/45f723e3cefd4f6b1bd5d40d84ff41a2063a9e71/arch/x86/include/asm/tlbflush.h [modify] https://crrev.com/45f723e3cefd4f6b1bd5d40d84ff41a2063a9e71/arch/x86/mm/tlb.c [modify] https://crrev.com/45f723e3cefd4f6b1bd5d40d84ff41a2063a9e71/mm/rmap.c
,
Nov 16 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromiumos/third_party/kernel/+/306de0a87139bda884fba977ddf0d576b905a601 commit 306de0a87139bda884fba977ddf0d576b905a601 Author: Ingo Molnar <mingo@kernel.org> Date: Thu Nov 16 19:38:57 2017 UPSTREAM: mm/mmu_context, sched/core: Fix mmu_context.h assumption Some architectures (such as Alpha) rely on include/linux/sched.h definitions in their mmu_context.h files. So include sched.h before mmu_context.h. BUG=chromium:779077 TEST=Build, boot on soraka. Cc: Andy Lutomirski <luto@kernel.org> Cc: Borislav Petkov <bp@alien8.de> Cc: Linus Torvalds <torvalds@linux-foundation.org> Cc: linux-kernel@vger.kernel.org Cc: Peter Zijlstra <peterz@infradead.org> Cc: Thomas Gleixner <tglx@linutronix.de> Signed-off-by: Ingo Molnar <mingo@kernel.org> (cherry picked from commit 8efd755ac2fe262d4c8d5c9bbe054bb67dae93da) Change-Id: I59ef1d786b1ec996cff98355325c39e47ce5ca69 Reviewed-on: https://chromium-review.googlesource.com/741901 Commit-Ready: Jorge Lucangeli Obes <jorgelo@chromium.org> Tested-by: Jorge Lucangeli Obes <jorgelo@chromium.org> Reviewed-by: Guenter Roeck <groeck@chromium.org> (cherry picked from commit 8246978a5ff7f100d403aa07f24e9cd69388d522) Signed-off-by: Daniel Wang <wonderfly@google.com> Reviewed-on: https://chromium-review.googlesource.com/764928 Reviewed-by: Bernie Thompson <bhthompson@chromium.org> [modify] https://crrev.com/306de0a87139bda884fba977ddf0d576b905a601/mm/mmu_context.c
,
Nov 16 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromiumos/third_party/kernel/+/69f856b27f45a64db76c4b4888a8241c36c4d084 commit 69f856b27f45a64db76c4b4888a8241c36c4d084 Author: Andy Lutomirski <luto@kernel.org> Date: Thu Nov 16 19:39:01 2017 UPSTREAM: sched/core: Add switch_mm_irqs_off() and use it in the scheduler By default, this is the same thing as switch_mm(). x86 will override it as an optimization. BUG=chromium:779077 TEST=Build, boot on soraka. Signed-off-by: Andy Lutomirski <luto@kernel.org> Reviewed-by: Borislav Petkov <bp@suse.de> Cc: Borislav Petkov <bp@alien8.de> Cc: Linus Torvalds <torvalds@linux-foundation.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Thomas Gleixner <tglx@linutronix.de> Link: http://lkml.kernel.org/r/df401df47bdd6be3e389c6f1e3f5310d70e81b2c.1461688545.git.luto@kernel.org Signed-off-by: Ingo Molnar <mingo@kernel.org> (cherry picked from commit f98db6013c557c216da5038d9c52045be55cd039) Change-Id: Ice480e7d97c49fca442dbcb2ff32861055ef8ddb Reviewed-on: https://chromium-review.googlesource.com/741902 Commit-Ready: Jorge Lucangeli Obes <jorgelo@chromium.org> Tested-by: Jorge Lucangeli Obes <jorgelo@chromium.org> Reviewed-by: Guenter Roeck <groeck@chromium.org> (cherry picked from commit 1d00ad67ff6c91329e4d16f6d943d877429fcf9e) Signed-off-by: Daniel Wang <wonderfly@google.com> Reviewed-on: https://chromium-review.googlesource.com/764929 Reviewed-by: Bernie Thompson <bhthompson@chromium.org> [modify] https://crrev.com/69f856b27f45a64db76c4b4888a8241c36c4d084/include/linux/mmu_context.h [modify] https://crrev.com/69f856b27f45a64db76c4b4888a8241c36c4d084/kernel/sched/core.c
,
Nov 16 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromiumos/third_party/kernel/+/0e9a480860defb2f60023e5939d4da4ffa83fc94 commit 0e9a480860defb2f60023e5939d4da4ffa83fc94 Author: Andy Lutomirski <luto@kernel.org> Date: Thu Nov 16 19:39:05 2017 UPSTREAM: x86/mm: Build arch/x86/mm/tlb.c even on !SMP Currently all of the functions that live in tlb.c are inlined on !SMP builds. One can debate whether this is a good idea (in many respects the code in tlb.c is better than the inlined UP code). Regardless, I want to add code that needs to be built on UP and SMP kernels and relates to tlb flushing, so arrange for tlb.c to be compiled unconditionally. BUG=chromium:779077 TEST=Build, boot on soraka. Signed-off-by: Andy Lutomirski <luto@kernel.org> Reviewed-by: Borislav Petkov <bp@suse.de> Cc: Borislav Petkov <bp@alien8.de> Cc: Linus Torvalds <torvalds@linux-foundation.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Thomas Gleixner <tglx@linutronix.de> Link: http://lkml.kernel.org/r/f0d778f0d828fc46e5d1946bca80f0aaf9abf032.1461688545.git.luto@kernel.org Signed-off-by: Ingo Molnar <mingo@kernel.org> (cherry picked from commit e1074888c326038340a1ada9129d679e661f2ea6) Change-Id: If7e0e98783e4c7ced3224e0ef65c9f162468947c Reviewed-on: https://chromium-review.googlesource.com/741903 Commit-Ready: Jorge Lucangeli Obes <jorgelo@chromium.org> Tested-by: Jorge Lucangeli Obes <jorgelo@chromium.org> Reviewed-by: Guenter Roeck <groeck@chromium.org> (cherry picked from commit 2ee9dcbaa02b3c381040711d0348b68ff8025827) Signed-off-by: Daniel Wang <wonderfly@google.com> Reviewed-on: https://chromium-review.googlesource.com/764930 Reviewed-by: Bernie Thompson <bhthompson@chromium.org> [modify] https://crrev.com/0e9a480860defb2f60023e5939d4da4ffa83fc94/arch/x86/mm/tlb.c [modify] https://crrev.com/0e9a480860defb2f60023e5939d4da4ffa83fc94/arch/x86/mm/Makefile
,
Nov 16 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromiumos/third_party/kernel/+/54c8658952db1c9a08d8f0bc2f8d1c49b7c6ee24 commit 54c8658952db1c9a08d8f0bc2f8d1c49b7c6ee24 Author: Andy Lutomirski <luto@kernel.org> Date: Thu Nov 16 19:39:09 2017 BACKPORT: x86/mm, sched/core: Uninline switch_mm() It's fairly large and it has quite a few callers. This may also help untangle some headers down the road. BUG=chromium:779077 TEST=Build, boot on soraka. Signed-off-by: Andy Lutomirski <luto@kernel.org> Reviewed-by: Borislav Petkov <bp@suse.de> Cc: Borislav Petkov <bp@alien8.de> Cc: Linus Torvalds <torvalds@linux-foundation.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Thomas Gleixner <tglx@linutronix.de> Link: http://lkml.kernel.org/r/54f3367803e7f80b2be62c8a21879aa74b1a5f57.1461688545.git.luto@kernel.org Signed-off-by: Ingo Molnar <mingo@kernel.org> (cherry picked from commit 69c0319aabba45bcf33178916a2f06967b4adede) Conflicts: arch/x86/include/asm/mmu_context.h Change-Id: Ib702286e2bedd74723708d0518a07d2cf5034c94 Reviewed-on: https://chromium-review.googlesource.com/741904 Commit-Ready: Jorge Lucangeli Obes <jorgelo@chromium.org> Tested-by: Jorge Lucangeli Obes <jorgelo@chromium.org> Reviewed-by: Guenter Roeck <groeck@chromium.org> (cherry picked from commit e311e1aca74be051a8ff7d6ca3c53a62137d2722) Signed-off-by: Daniel Wang <wonderfly@google.com> Reviewed-on: https://chromium-review.googlesource.com/764931 Reviewed-by: Bernie Thompson <bhthompson@chromium.org> [modify] https://crrev.com/54c8658952db1c9a08d8f0bc2f8d1c49b7c6ee24/arch/x86/include/asm/mmu_context.h [modify] https://crrev.com/54c8658952db1c9a08d8f0bc2f8d1c49b7c6ee24/arch/x86/mm/tlb.c
,
Nov 16 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromiumos/third_party/kernel/+/922bc1e22915bfef86f7f30fedf159404c52077d commit 922bc1e22915bfef86f7f30fedf159404c52077d Author: Andy Lutomirski <luto@kernel.org> Date: Thu Nov 16 19:39:13 2017 UPSTREAM: x86/mm, sched/core: Turn off IRQs in switch_mm() Potential races between switch_mm() and TLB-flush or LDT-flush IPIs could be very messy. AFAICT the code is currently okay, whether by accident or by careful design, but enabling PCID will make it considerably more complicated and will no longer be obviously safe. Fix it with a big hammer: run switch_mm() with IRQs off. To avoid a performance hit in the scheduler, we take advantage of our knowledge that the scheduler already has IRQs disabled when it calls switch_mm(). BUG=chromium:779077 TEST=Build, boot on soraka. Signed-off-by: Andy Lutomirski <luto@kernel.org> Reviewed-by: Borislav Petkov <bp@suse.de> Cc: Borislav Petkov <bp@alien8.de> Cc: Linus Torvalds <torvalds@linux-foundation.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Thomas Gleixner <tglx@linutronix.de> Link: http://lkml.kernel.org/r/f19baf759693c9dcae64bbff76189db77cb13398.1461688545.git.luto@kernel.org Signed-off-by: Ingo Molnar <mingo@kernel.org> (cherry picked from commit 078194f8e9fe3cf54c8fd8bded48a1db5bd8eb8a) Change-Id: Ie46238916f004d6a5e81fc03c0701cd187745ef7 Reviewed-on: https://chromium-review.googlesource.com/741905 Commit-Ready: Jorge Lucangeli Obes <jorgelo@chromium.org> Tested-by: Jorge Lucangeli Obes <jorgelo@chromium.org> Reviewed-by: Guenter Roeck <groeck@chromium.org> (cherry picked from commit e1c536b810740c75d7463f16f317e2b486f041af) Signed-off-by: Daniel Wang <wonderfly@google.com> Reviewed-on: https://chromium-review.googlesource.com/764932 Reviewed-by: Bernie Thompson <bhthompson@chromium.org> [modify] https://crrev.com/922bc1e22915bfef86f7f30fedf159404c52077d/arch/x86/include/asm/mmu_context.h [modify] https://crrev.com/922bc1e22915bfef86f7f30fedf159404c52077d/arch/x86/mm/tlb.c
,
Nov 16 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromiumos/third_party/kernel/+/00755440a8fdd1906247b62a3f3d0ee07abf83c1 commit 00755440a8fdd1906247b62a3f3d0ee07abf83c1 Author: Steven Rostedt <rostedt@goodmis.org> Date: Thu Nov 16 19:39:17 2017 UPSTREAM: ARM: Hide finish_arch_post_lock_switch() from modules The introduction of switch_mm_irqs_off() brought back an old bug regarding the use of preempt_enable_no_resched: As part of: 62b94a08da1b ("sched/preempt: Take away preempt_enable_no_resched() from modules") the definition of preempt_enable_no_resched() is only available in built-in code, not in loadable modules, so we can't generally use it from header files. However, the ARM version of finish_arch_post_lock_switch() calls preempt_enable_no_resched() and is defined as a static inline function in asm/mmu_context.h. This in turn means we cannot include asm/mmu_context.h from modules. With today's tip tree, asm/mmu_context.h gets included from linux/mmu_context.h, which is normally the exact pattern one would expect, but unfortunately, linux/mmu_context.h can be included from the vhost driver that is a loadable module, now causing this compile time error with modular configs: In file included from ../include/linux/mmu_context.h:4:0, from ../drivers/vhost/vhost.c:18: ../arch/arm/include/asm/mmu_context.h: In function 'finish_arch_post_lock_switch': ../arch/arm/include/asm/mmu_context.h:88:3: error: implicit declaration of function 'preempt_enable_no_resched' [-Werror=implicit-function-declaration] preempt_enable_no_resched(); Andy already tried to fix the bug by including linux/preempt.h from asm/mmu_context.h, but that didn't help. Arnd suggested reordering the header files, which wasn't popular, so let's use this workaround instead: The finish_arch_post_lock_switch() definition is now also hidden inside of #ifdef MODULE, so we don't see anything referencing preempt_enable_no_resched() from a header file. I've built a few hundred randconfig kernels with this, and did not see any new problems. BUG=chromium:779077 TEST=Build, boot on soraka. Tested-by: Guenter Roeck <linux@roeck-us.net> Signed-off-by: Steven Rostedt <rostedt@goodmis.org> Signed-off-by: Arnd Bergmann <arnd@arndb.de> Acked-by: Russell King <rmk+kernel@arm.linux.org.uk> Cc: Alexander Shishkin <alexander.shishkin@linux.intel.com> Cc: Andy Lutomirski <luto@amacapital.net> Cc: Andy Lutomirski <luto@kernel.org> Cc: Ard Biesheuvel <ard.biesheuvel@linaro.org> Cc: Arnaldo Carvalho de Melo <acme@redhat.com> Cc: Borislav Petkov <bp@suse.de> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Linus Torvalds <torvalds@linux-foundation.org> Cc: Mel Gorman <mgorman@techsingularity.net> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Russell King - ARM Linux <linux@armlinux.org.uk> Cc: Stephane Eranian <eranian@google.com> Cc: Thomas Gleixner <tglx@linutronix.de> Cc: Vince Weaver <vincent.weaver@maine.edu> Cc: linux-arm-kernel@lists.infradead.org Fixes: f98db6013c55 ("sched/core: Add switch_mm_irqs_off() and use it in the scheduler") Link: http://lkml.kernel.org/r/1463146234-161304-1-git-send-email-arnd@arndb.de Signed-off-by: Ingo Molnar <mingo@kernel.org> (cherry picked from commit ef0491ea17f8019821c7e9c8e801184ecf17f85a) Change-Id: I01861e082dea345e46152b2a06765e9634ac84fa Reviewed-on: https://chromium-review.googlesource.com/741906 Commit-Ready: Jorge Lucangeli Obes <jorgelo@chromium.org> Tested-by: Jorge Lucangeli Obes <jorgelo@chromium.org> Reviewed-by: Guenter Roeck <groeck@chromium.org> (cherry picked from commit 71cf954e4cf0c146a94a557dc2c6877f300928b1) Signed-off-by: Daniel Wang <wonderfly@google.com> Reviewed-on: https://chromium-review.googlesource.com/764933 Reviewed-by: Bernie Thompson <bhthompson@chromium.org> [modify] https://crrev.com/00755440a8fdd1906247b62a3f3d0ee07abf83c1/arch/arm/include/asm/mmu_context.h
,
Nov 16 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromiumos/third_party/kernel/+/ba6f3a80c935db7001cfc03db4639e79991a936c commit ba6f3a80c935db7001cfc03db4639e79991a936c Author: Andy Lutomirski <luto@kernel.org> Date: Thu Nov 16 19:39:21 2017 UPSTREAM: sched/core: Idle_task_exit() shouldn't use switch_mm_irqs_off() idle_task_exit() can be called with IRQs on x86 on and therefore should use switch_mm(), not switch_mm_irqs_off(). This doesn't seem to cause any problems right now, but it will confuse my upcoming TLB flush changes. Nonetheless, I think it should be backported because it's trivial. There won't be any meaningful performance impact because idle_task_exit() is only used when offlining a CPU. BUG=chromium:779077 TEST=Build, boot on soraka. Signed-off-by: Andy Lutomirski <luto@kernel.org> Cc: Borislav Petkov <bp@suse.de> Cc: Linus Torvalds <torvalds@linux-foundation.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Thomas Gleixner <tglx@linutronix.de> Cc: stable@vger.kernel.org Fixes: f98db6013c55 ("sched/core: Add switch_mm_irqs_off() and use it in the scheduler") Link: http://lkml.kernel.org/r/ca3d1a9fa93a0b49f5a8ff729eda3640fb6abdf9.1497034141.git.luto@kernel.org Signed-off-by: Ingo Molnar <mingo@kernel.org> (cherry picked from commit 252d2a4117bc181b287eeddf848863788da733ae) Change-Id: I65b268b1c8654b0df554abdca3d2290fba8a94b3 Reviewed-on: https://chromium-review.googlesource.com/741907 Commit-Ready: Jorge Lucangeli Obes <jorgelo@chromium.org> Tested-by: Jorge Lucangeli Obes <jorgelo@chromium.org> Reviewed-by: Guenter Roeck <groeck@chromium.org> (cherry picked from commit 58ee5e48938005bbcb31776b59a7858db3813238) Signed-off-by: Daniel Wang <wonderfly@google.com> Reviewed-on: https://chromium-review.googlesource.com/764934 Reviewed-by: Bernie Thompson <bhthompson@chromium.org> [modify] https://crrev.com/ba6f3a80c935db7001cfc03db4639e79991a936c/kernel/sched/core.c
,
Nov 16 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromiumos/third_party/kernel/+/a31c558f3cbedb98a91748e9d1d9dae2ebbd2444 commit a31c558f3cbedb98a91748e9d1d9dae2ebbd2444 Author: Aaron Lu <aaron.lu@intel.com> Date: Thu Nov 16 19:39:25 2017 UPSTREAM: x86/irq: Do not substract irq_tlb_count from irq_call_count Since commit: 52aec3308db8 ("x86/tlb: replace INVALIDATE_TLB_VECTOR by CALL_FUNCTION_VECTOR") the TLB remote shootdown is done through call function vector. That commit didn't take care of irq_tlb_count, which a later commit: fd0f5869724f ("x86: Distinguish TLB shootdown interrupts from other functions call interrupts") ... tried to fix. The fix assumes every increase of irq_tlb_count has a corresponding increase of irq_call_count. So the irq_call_count is always bigger than irq_tlb_count and we could substract irq_tlb_count from irq_call_count. Unfortunately this is not true for the smp_call_function_single() case. The IPI is only sent if the target CPU's call_single_queue is empty when adding a csd into it in generic_exec_single. That means if two threads are both adding flush tlb csds to the same CPU's call_single_queue, only one IPI is sent. In other words, the irq_call_count is incremented by 1 but irq_tlb_count is incremented by 2. Over time, irq_tlb_count will be bigger than irq_call_count and the substract will produce a very large irq_call_count value due to overflow. Considering that: 1) it's not worth to send more IPIs for the sake of accurate counting of irq_call_count in generic_exec_single(); 2) it's not easy to tell if the call function interrupt is for TLB shootdown in __smp_call_function_single_interrupt(). Not to exclude TLB shootdown from call function count seems to be the simplest fix and this patch just does that. This bug was found by LKP's cyclic performance regression tracking recently with the vm-scalability test suite. I have bisected to commit: 3dec0ba0be6a ("mm/rmap: share the i_mmap_rwsem") This commit didn't do anything wrong but revealed the irq_call_count problem. IIUC, the commit makes rwc->remap_one in rmap_walk_file concurrent with multiple threads. When remap_one is try_to_unmap_one(), then multiple threads could queue flush TLB to the same CPU but only one IPI will be sent. Since the commit was added in Linux v3.19, the counting problem only shows up from v3.19 onwards. BUG=chromium:779077 TEST=Build, boot on soraka. Signed-off-by: Aaron Lu <aaron.lu@intel.com> Cc: Alex Shi <alex.shi@linaro.org> Cc: Andy Lutomirski <luto@kernel.org> Cc: Borislav Petkov <bp@alien8.de> Cc: Brian Gerst <brgerst@gmail.com> Cc: Davidlohr Bueso <dave@stgolabs.net> Cc: Denys Vlasenko <dvlasenk@redhat.com> Cc: H. Peter Anvin <hpa@zytor.com> Cc: Huang Ying <ying.huang@intel.com> Cc: Josh Poimboeuf <jpoimboe@redhat.com> Cc: Linus Torvalds <torvalds@linux-foundation.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Thomas Gleixner <tglx@linutronix.de> Cc: Tomoki Sekiyama <tomoki.sekiyama.qu@hitachi.com> Link: http://lkml.kernel.org/r/20160811074430.GA18163@aaronlu.sh.intel.com Signed-off-by: Ingo Molnar <mingo@kernel.org> (cherry picked from commit 82ba4faca1bffad429f15c90c980ffd010366c25) Change-Id: Id2795ea61eaa127bf90b2d06ee8fe1c01fd1ca5b Reviewed-on: https://chromium-review.googlesource.com/741908 Commit-Ready: Jorge Lucangeli Obes <jorgelo@chromium.org> Tested-by: Jorge Lucangeli Obes <jorgelo@chromium.org> Reviewed-by: Guenter Roeck <groeck@chromium.org> (cherry picked from commit 7c99f796caea737c792c7ad4aba2cab9cfa929f1) Signed-off-by: Daniel Wang <wonderfly@google.com> Reviewed-on: https://chromium-review.googlesource.com/764935 Reviewed-by: Bernie Thompson <bhthompson@chromium.org> [modify] https://crrev.com/a31c558f3cbedb98a91748e9d1d9dae2ebbd2444/arch/x86/kernel/irq.c [modify] https://crrev.com/a31c558f3cbedb98a91748e9d1d9dae2ebbd2444/arch/x86/include/asm/hardirq.h
,
Nov 16 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromiumos/third_party/kernel/+/25f31c0b5322efe52ac831d18cd02b7f574ea553 commit 25f31c0b5322efe52ac831d18cd02b7f574ea553 Author: Andy Lutomirski <luto@kernel.org> Date: Thu Nov 16 19:39:29 2017 UPSTREAM: x86/vm86/32: Switch to flush_tlb_mm_range() in mark_screen_rdonly() mark_screen_rdonly() is the last remaining caller of flush_tlb(). flush_tlb_mm_range() is potentially faster and isn't obsolete. Compile-tested only because I don't know whether software that uses this mechanism even exists. BUG=chromium:779077 TEST=Build, boot on soraka. Signed-off-by: Andy Lutomirski <luto@kernel.org> Cc: Andrew Morton <akpm@linux-foundation.org> Cc: Borislav Petkov <bp@alien8.de> Cc: Brian Gerst <brgerst@gmail.com> Cc: Dave Hansen <dave.hansen@intel.com> Cc: Denys Vlasenko <dvlasenk@redhat.com> Cc: H. Peter Anvin <hpa@zytor.com> Cc: Josh Poimboeuf <jpoimboe@redhat.com> Cc: Linus Torvalds <torvalds@linux-foundation.org> Cc: Michal Hocko <mhocko@suse.com> Cc: Nadav Amit <namit@vmware.com> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Rik van Riel <riel@redhat.com> Cc: Sasha Levin <sasha.levin@oracle.com> Cc: Thomas Gleixner <tglx@linutronix.de> Link: http://lkml.kernel.org/r/791a644076fc3577ba7f7b7cafd643cc089baa7d.1492844372.git.luto@kernel.org Signed-off-by: Ingo Molnar <mingo@kernel.org> (cherry picked from commit 9ccee2373f0658f234727700e619df097ba57023) Change-Id: I1a1e0f9276cc0856624f6af434c050ae36c78757 Reviewed-on: https://chromium-review.googlesource.com/741909 Commit-Ready: Jorge Lucangeli Obes <jorgelo@chromium.org> Tested-by: Jorge Lucangeli Obes <jorgelo@chromium.org> Reviewed-by: Guenter Roeck <groeck@chromium.org> (cherry picked from commit 48b90532c61b8b1421a8756032f885b651e9574b) Signed-off-by: Daniel Wang <wonderfly@google.com> Reviewed-on: https://chromium-review.googlesource.com/764936 Reviewed-by: Bernie Thompson <bhthompson@chromium.org> [modify] https://crrev.com/25f31c0b5322efe52ac831d18cd02b7f574ea553/arch/x86/kernel/vm86_32.c
,
Nov 16 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromiumos/third_party/kernel/+/6c66a6ec9bccd92e48789aefd48f4f864ba6a7f2 commit 6c66a6ec9bccd92e48789aefd48f4f864ba6a7f2 Author: Andy Lutomirski <luto@kernel.org> Date: Thu Nov 16 19:39:34 2017 UPSTREAM: x86/mm: Remove flush_tlb() and flush_tlb_current_task() I was trying to figure out what how flush_tlb_current_task() would possibly work correctly if current->mm != current->active_mm, but I realized I could spare myself the effort: it has no callers except the unused flush_tlb() macro. BUG=chromium:779077 TEST=Build, boot on soraka. Signed-off-by: Andy Lutomirski <luto@kernel.org> Cc: Andrew Morton <akpm@linux-foundation.org> Cc: Borislav Petkov <bp@alien8.de> Cc: Brian Gerst <brgerst@gmail.com> Cc: Dave Hansen <dave.hansen@intel.com> Cc: Denys Vlasenko <dvlasenk@redhat.com> Cc: H. Peter Anvin <hpa@zytor.com> Cc: Josh Poimboeuf <jpoimboe@redhat.com> Cc: Linus Torvalds <torvalds@linux-foundation.org> Cc: Michal Hocko <mhocko@suse.com> Cc: Nadav Amit <namit@vmware.com> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Rik van Riel <riel@redhat.com> Cc: Thomas Gleixner <tglx@linutronix.de> Link: http://lkml.kernel.org/r/e52d64c11690f85e9f1d69d7b48cc2269cd2e94b.1492844372.git.luto@kernel.org Signed-off-by: Ingo Molnar <mingo@kernel.org> (cherry picked from commit 29961b59a51f8c6838a26a45e871a7ed6771809b) Change-Id: Ie60b99e6c492e5557190c9d730143b90a667ece3 Reviewed-on: https://chromium-review.googlesource.com/741910 Commit-Ready: Jorge Lucangeli Obes <jorgelo@chromium.org> Tested-by: Jorge Lucangeli Obes <jorgelo@chromium.org> Reviewed-by: Guenter Roeck <groeck@chromium.org> (cherry picked from commit 583dc8e9bfed2d15102deb4ad105872d9c410020) Signed-off-by: Daniel Wang <wonderfly@google.com> Reviewed-on: https://chromium-review.googlesource.com/764937 Reviewed-by: Bernie Thompson <bhthompson@chromium.org> [modify] https://crrev.com/6c66a6ec9bccd92e48789aefd48f4f864ba6a7f2/arch/x86/include/asm/tlbflush.h [modify] https://crrev.com/6c66a6ec9bccd92e48789aefd48f4f864ba6a7f2/arch/x86/mm/tlb.c
,
Nov 16 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromiumos/third_party/kernel/+/bd2307fe65751c224243385e1e0352427c3c3d30 commit bd2307fe65751c224243385e1e0352427c3c3d30 Author: Andy Lutomirski <luto@kernel.org> Date: Thu Nov 16 19:39:38 2017 UPSTREAM: x86/mm: Make flush_tlb_mm_range() more predictable I'm about to rewrite the function almost completely, but first I want to get a functional change out of the way. Currently, if flush_tlb_mm_range() does not flush the local TLB at all, it will never do individual page flushes on remote CPUs. This seems to be an accident, and preserving it will be awkward. Let's change it first so that any regressions in the rewrite will be easier to bisect and so that the rewrite can attempt to change no visible behavior at all. The fix is simple: we can simply avoid short-circuiting the calculation of base_pages_to_flush. As a side effect, this also eliminates a potential corner case: if tlb_single_page_flush_ceiling == TLB_FLUSH_ALL, flush_tlb_mm_range() could have ended up flushing the entire address space one page at a time. BUG=chromium:779077 TEST=Build, boot on soraka. Signed-off-by: Andy Lutomirski <luto@kernel.org> Acked-by: Dave Hansen <dave.hansen@intel.com> Cc: Andrew Morton <akpm@linux-foundation.org> Cc: Borislav Petkov <bp@alien8.de> Cc: Brian Gerst <brgerst@gmail.com> Cc: Denys Vlasenko <dvlasenk@redhat.com> Cc: H. Peter Anvin <hpa@zytor.com> Cc: Josh Poimboeuf <jpoimboe@redhat.com> Cc: Linus Torvalds <torvalds@linux-foundation.org> Cc: Michal Hocko <mhocko@suse.com> Cc: Nadav Amit <namit@vmware.com> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Rik van Riel <riel@redhat.com> Cc: Thomas Gleixner <tglx@linutronix.de> Link: http://lkml.kernel.org/r/4b29b771d9975aad7154c314534fec235618175a.1492844372.git.luto@kernel.org Signed-off-by: Ingo Molnar <mingo@kernel.org> (cherry picked from commit ce27374fabf553153c3f53efcaa9bfab9216bd8c) Change-Id: If0567ae7a6c629e332605f907af7f11332106bef Reviewed-on: https://chromium-review.googlesource.com/741911 Commit-Ready: Jorge Lucangeli Obes <jorgelo@chromium.org> Tested-by: Jorge Lucangeli Obes <jorgelo@chromium.org> Reviewed-by: Guenter Roeck <groeck@chromium.org> (cherry picked from commit d8a45e7a6989d7b6e1302c254888aeb560a1103c) Signed-off-by: Daniel Wang <wonderfly@google.com> Reviewed-on: https://chromium-review.googlesource.com/764938 Reviewed-by: Bernie Thompson <bhthompson@chromium.org> [modify] https://crrev.com/bd2307fe65751c224243385e1e0352427c3c3d30/arch/x86/mm/tlb.c
,
Nov 16 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromiumos/third_party/kernel/+/5b51bd2ad9a0236a1af7d8d809d3abab13575caa commit 5b51bd2ad9a0236a1af7d8d809d3abab13575caa Author: Andy Lutomirski <luto@kernel.org> Date: Thu Nov 16 19:39:42 2017 UPSTREAM: x86/mm: Reimplement flush_tlb_page() using flush_tlb_mm_range() flush_tlb_page() was very similar to flush_tlb_mm_range() except that it had a couple of issues: - It was missing an smp_mb() in the case where current->active_mm != mm. (This is a longstanding bug reported by Nadav Amit) - It was missing tracepoints and vm counter updates. The only reason that I can see for keeping it at as a separate function is that it could avoid a few branches that flush_tlb_mm_range() needs to decide to flush just one page. This hardly seems worthwhile. If we decide we want to get rid of those branches again, a better way would be to introduce an __flush_tlb_mm_range() helper and make both flush_tlb_page() and flush_tlb_mm_range() use it. BUG=chromium:779077 TEST=Build, boot on soraka. Signed-off-by: Andy Lutomirski <luto@kernel.org> Acked-by: Kees Cook <keescook@chromium.org> Cc: Andrew Morton <akpm@linux-foundation.org> Cc: Borislav Petkov <bpetkov@suse.de> Cc: Dave Hansen <dave.hansen@intel.com> Cc: Linus Torvalds <torvalds@linux-foundation.org> Cc: Mel Gorman <mgorman@suse.de> Cc: Michal Hocko <mhocko@suse.com> Cc: Nadav Amit <nadav.amit@gmail.com> Cc: Nadav Amit <namit@vmware.com> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Rik van Riel <riel@redhat.com> Cc: Thomas Gleixner <tglx@linutronix.de> Cc: linux-mm@kvack.org Link: http://lkml.kernel.org/r/3cc3847cf888d8907577569b8bac3f01992ef8f9.1495492063.git.luto@kernel.org Signed-off-by: Ingo Molnar <mingo@kernel.org> (cherry picked from commit ca6c99c0794875c6d1db6e22f246699691ab7e6b) Change-Id: I681666b8c6d94a8d6d1aa177136fafc206c28ccb Reviewed-on: https://chromium-review.googlesource.com/741912 Commit-Ready: Jorge Lucangeli Obes <jorgelo@chromium.org> Tested-by: Jorge Lucangeli Obes <jorgelo@chromium.org> Reviewed-by: Guenter Roeck <groeck@chromium.org> (cherry picked from commit 1b53b4d133ea4dcff396590fb28262e930d694cb) Signed-off-by: Daniel Wang <wonderfly@google.com> Reviewed-on: https://chromium-review.googlesource.com/764939 Reviewed-by: Bernie Thompson <bhthompson@chromium.org> [modify] https://crrev.com/5b51bd2ad9a0236a1af7d8d809d3abab13575caa/arch/x86/include/asm/tlbflush.h [modify] https://crrev.com/5b51bd2ad9a0236a1af7d8d809d3abab13575caa/arch/x86/mm/tlb.c
,
Nov 16 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromiumos/third_party/kernel/+/5f9b1965ad5584d1a24d95b3a07c13edd601e84d commit 5f9b1965ad5584d1a24d95b3a07c13edd601e84d Author: Andy Lutomirski <luto@kernel.org> Date: Thu Nov 16 19:39:46 2017 BACKPORT: x86/mm: Remove the UP asm/tlbflush.h code, always use the (formerly) SMP code The UP asm/tlbflush.h generates somewhat nicer code than the SMP version. Aside from that, it's fallen quite a bit behind the SMP code: - flush_tlb_mm_range() didn't flush individual pages if the range was small. - The lazy TLB code was much weaker. This usually wouldn't matter, but, if a kernel thread flushed its lazy "active_mm" more than once (due to reclaim or similar), it wouldn't be unlazied and would instead pointlessly flush repeatedly. - Tracepoints were missing. Aside from that, simply having the UP code around was a maintanence burden, since it means that any change to the TLB flush code had to make sure not to break it. Simplify everything by deleting the UP code. BUG=chromium:779077 TEST=Build, boot on soraka. Signed-off-by: Andy Lutomirski <luto@kernel.org> Cc: Andrew Morton <akpm@linux-foundation.org> Cc: Arjan van de Ven <arjan@linux.intel.com> Cc: Borislav Petkov <bpetkov@suse.de> Cc: Dave Hansen <dave.hansen@intel.com> Cc: Linus Torvalds <torvalds@linux-foundation.org> Cc: Mel Gorman <mgorman@suse.de> Cc: Michal Hocko <mhocko@suse.com> Cc: Nadav Amit <nadav.amit@gmail.com> Cc: Nadav Amit <namit@vmware.com> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Rik van Riel <riel@redhat.com> Cc: Thomas Gleixner <tglx@linutronix.de> Cc: linux-mm@kvack.org Signed-off-by: Ingo Molnar <mingo@kernel.org> (cherry picked from commit ce4a4e565f5264909a18c733b864c3f74467f69e) Conflicts: arch/x86/Kconfig arch/x86/include/asm/tlbbatch.h arch/x86/include/asm/tlbflush.h arch/x86/mm/tlb.c Change-Id: I71687765e75ac212e7eea239c5373aabf7b2bfbf Reviewed-on: https://chromium-review.googlesource.com/741913 Commit-Ready: Jorge Lucangeli Obes <jorgelo@chromium.org> Tested-by: Jorge Lucangeli Obes <jorgelo@chromium.org> Reviewed-by: Guenter Roeck <groeck@chromium.org> (cherry picked from commit b7c0e9d3714dc250d920264e8a9bddf3f701c4aa) Signed-off-by: Daniel Wang <wonderfly@google.com> Reviewed-on: https://chromium-review.googlesource.com/764940 Reviewed-by: Bernie Thompson <bhthompson@chromium.org> [modify] https://crrev.com/5f9b1965ad5584d1a24d95b3a07c13edd601e84d/arch/x86/mm/tlb.c [modify] https://crrev.com/5f9b1965ad5584d1a24d95b3a07c13edd601e84d/arch/x86/include/asm/mmu_context.h [modify] https://crrev.com/5f9b1965ad5584d1a24d95b3a07c13edd601e84d/arch/x86/include/asm/hardirq.h [modify] https://crrev.com/5f9b1965ad5584d1a24d95b3a07c13edd601e84d/arch/x86/mm/init.c [modify] https://crrev.com/5f9b1965ad5584d1a24d95b3a07c13edd601e84d/arch/x86/include/asm/tlbflush.h [modify] https://crrev.com/5f9b1965ad5584d1a24d95b3a07c13edd601e84d/arch/x86/Kconfig [modify] https://crrev.com/5f9b1965ad5584d1a24d95b3a07c13edd601e84d/arch/x86/include/asm/mmu.h
,
Nov 16 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromiumos/third_party/kernel/+/9d66f0fbcb1b6e2f93dd06d5948c6788f38c2f67 commit 9d66f0fbcb1b6e2f93dd06d5948c6788f38c2f67 Author: Andy Lutomirski <luto@kernel.org> Date: Thu Nov 16 19:39:50 2017 UPSTREAM: x86/mm: Disable PCID on 32-bit kernels 32-bit kernels on new hardware will see PCID in CPUID, but PCID can only be used in 64-bit mode. Rather than making all PCID code conditional, just disable the feature on 32-bit builds. BUG=chromium:779077 TEST=Build, boot on soraka. Signed-off-by: Andy Lutomirski <luto@kernel.org> Reviewed-by: Nadav Amit <nadav.amit@gmail.com> Reviewed-by: Borislav Petkov <bp@suse.de> Reviewed-by: Thomas Gleixner <tglx@linutronix.de> Cc: Andrew Morton <akpm@linux-foundation.org> Cc: Arjan van de Ven <arjan@linux.intel.com> Cc: Borislav Petkov <bp@alien8.de> Cc: Dave Hansen <dave.hansen@intel.com> Cc: Linus Torvalds <torvalds@linux-foundation.org> Cc: Mel Gorman <mgorman@suse.de> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Rik van Riel <riel@redhat.com> Cc: linux-mm@kvack.org Link: http://lkml.kernel.org/r/2e391769192a4d31b808410c383c6bf0734bc6ea.1498751203.git.luto@kernel.org Signed-off-by: Ingo Molnar <mingo@kernel.org> (cherry picked from commit cba4671af7550e008f7a7835f06df0763825bf3e) Change-Id: I2f91901f636958a0e19e2cfd02e7d6e143a9d3d0 Reviewed-on: https://chromium-review.googlesource.com/741914 Commit-Ready: Jorge Lucangeli Obes <jorgelo@chromium.org> Tested-by: Jorge Lucangeli Obes <jorgelo@chromium.org> Reviewed-by: Guenter Roeck <groeck@chromium.org> (cherry picked from commit 5929d58eab3898fdb995c77d339930498a3b926d) Signed-off-by: Daniel Wang <wonderfly@google.com> Reviewed-on: https://chromium-review.googlesource.com/764941 Reviewed-by: Bernie Thompson <bhthompson@chromium.org> [modify] https://crrev.com/9d66f0fbcb1b6e2f93dd06d5948c6788f38c2f67/arch/x86/include/asm/disabled-features.h [modify] https://crrev.com/9d66f0fbcb1b6e2f93dd06d5948c6788f38c2f67/arch/x86/kernel/cpu/bugs.c
,
Nov 16 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromiumos/third_party/kernel/+/682b2b7e5db19c85983b09ce46089f6fa304419d commit 682b2b7e5db19c85983b09ce46089f6fa304419d Author: Andy Lutomirski <luto@kernel.org> Date: Thu Nov 16 19:39:54 2017 BACKPORT: x86/mm: Add the 'nopcid' boot option to turn off PCID The parameter is only present on x86_64 systems to save a few bytes, as PCID is always disabled on x86_32. BUG=chromium:779077 TEST=Build, boot on soraka. Signed-off-by: Andy Lutomirski <luto@kernel.org> Reviewed-by: Nadav Amit <nadav.amit@gmail.com> Reviewed-by: Borislav Petkov <bp@suse.de> Reviewed-by: Thomas Gleixner <tglx@linutronix.de> Cc: Andrew Morton <akpm@linux-foundation.org> Cc: Arjan van de Ven <arjan@linux.intel.com> Cc: Borislav Petkov <bp@alien8.de> Cc: Dave Hansen <dave.hansen@intel.com> Cc: Linus Torvalds <torvalds@linux-foundation.org> Cc: Mel Gorman <mgorman@suse.de> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Rik van Riel <riel@redhat.com> Cc: linux-mm@kvack.org Link: http://lkml.kernel.org/r/8bbb2e65bcd249a5f18bfb8128b4689f08ac2b60.1498751203.git.luto@kernel.org Signed-off-by: Ingo Molnar <mingo@kernel.org> (cherry picked from commit 0790c9aad84901ca1bdc14746175549c8b5da215) Conflicts: Documentation/admin-guide/kernel-parameters.txt (not in this tree) Documentation/kernel-parameters.txt (patched instead of that) Change-Id: I439569f5b200a1d3d04e46ac0e0b4d49aeefe5f9 Reviewed-on: https://chromium-review.googlesource.com/741915 Commit-Ready: Jorge Lucangeli Obes <jorgelo@chromium.org> Tested-by: Jorge Lucangeli Obes <jorgelo@chromium.org> Reviewed-by: Guenter Roeck <groeck@chromium.org> (cherry picked from commit d0b2a4feb02ae78a3dcc621a0e126feb417cb82a) Signed-off-by: Daniel Wang <wonderfly@google.com> Reviewed-on: https://chromium-review.googlesource.com/764942 Reviewed-by: Bernie Thompson <bhthompson@chromium.org> [modify] https://crrev.com/682b2b7e5db19c85983b09ce46089f6fa304419d/Documentation/kernel-parameters.txt [modify] https://crrev.com/682b2b7e5db19c85983b09ce46089f6fa304419d/arch/x86/kernel/cpu/common.c
,
Nov 16 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromiumos/third_party/kernel/+/90912da3a3a76819fc30b5f8d50824f2a6f03f30 commit 90912da3a3a76819fc30b5f8d50824f2a6f03f30 Author: Andy Lutomirski <luto@kernel.org> Date: Thu Nov 16 19:39:58 2017 BACKPORT: x86/mm: Enable CR4.PCIDE on supported systems We can use PCID if the CPU has PCID and PGE and we're not on Xen. By itself, this has no effect. A followup patch will start using PCID. BUG=chromium:779077 TEST=Build, boot on soraka. Signed-off-by: Andy Lutomirski <luto@kernel.org> Reviewed-by: Nadav Amit <nadav.amit@gmail.com> Reviewed-by: Boris Ostrovsky <boris.ostrovsky@oracle.com> Reviewed-by: Thomas Gleixner <tglx@linutronix.de> Cc: Andrew Morton <akpm@linux-foundation.org> Cc: Arjan van de Ven <arjan@linux.intel.com> Cc: Borislav Petkov <bp@alien8.de> Cc: Dave Hansen <dave.hansen@intel.com> Cc: Juergen Gross <jgross@suse.com> Cc: Linus Torvalds <torvalds@linux-foundation.org> Cc: Mel Gorman <mgorman@suse.de> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Rik van Riel <riel@redhat.com> Cc: linux-mm@kvack.org Link: http://lkml.kernel.org/r/6327ecd907b32f79d5aa0d466f04503bbec5df88.1498751203.git.luto@kernel.org Signed-off-by: Ingo Molnar <mingo@kernel.org> (cherry picked from commit 660da7c9228f685b2ebe664f9fd69aaddcc420b5) Conflicts: arch/x86/xen/enlighten_pv.c (not in this tree) arch/x86/xen/enlighten.c (patched instead of that) Change-Id: I4a0991e84f87d875b796981c73d87aee5f452ee0 Reviewed-on: https://chromium-review.googlesource.com/741916 Commit-Ready: Jorge Lucangeli Obes <jorgelo@chromium.org> Tested-by: Jorge Lucangeli Obes <jorgelo@chromium.org> Reviewed-by: Guenter Roeck <groeck@chromium.org> (cherry picked from commit 82a9a43487eeb9ef47ad13a7d314de35c68e9199) Signed-off-by: Daniel Wang <wonderfly@google.com> Reviewed-on: https://chromium-review.googlesource.com/764943 Reviewed-by: Bernie Thompson <bhthompson@chromium.org> [modify] https://crrev.com/90912da3a3a76819fc30b5f8d50824f2a6f03f30/arch/x86/include/asm/tlbflush.h [modify] https://crrev.com/90912da3a3a76819fc30b5f8d50824f2a6f03f30/arch/x86/xen/enlighten.c [modify] https://crrev.com/90912da3a3a76819fc30b5f8d50824f2a6f03f30/arch/x86/kernel/cpu/common.c
,
Nov 22 2017
|
|||||||||||||||
►
Sign in to add a comment |
|||||||||||||||
Comment 1 by bugdroid1@chromium.org
, Oct 27 2017