New issue
Advanced search Search tips

Issue 922254 link

Starred by 2 users

Issue metadata

Status: Untriaged
Owner:
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: Chrome
Pri: 1
Type: Bug



Sign in to add a comment

tast.platform.ChromeMlocked fails on peach_pit-tot-chrome-pfq-informational

Project Member Reported by afakhry@google.com, Jan 15

Issue description

- peach_pit-tot-chrome-pfq-informational: https://cros-goldeneye.corp.google.com/chromeos/healthmonitoring/buildDetails?buildbucketId=8924242727641122208

tast.platform.ChromeMlocked               FAIL: Checking processes failed: context deadline exceeded; last error follows: no mlocked pages found; checked procs []int32{29607, 29642, 29661, 29687, 29702, 30047, 30173, 30174, 30288, 30487}

+gbiv@ It seems you wrote this test? Do you have an idea what's going on here?

There is a bunch of crash dumps but the stack traces are corrupted:

https://00e9e64bac39096e4be6725a59ffed784ad44a63ca58c189c3-apidata.googleusercontent.com/download/storage/v1/b/chromeos-autotest-results/o/277251581-chromeos-test%2Fchromeos6-row2-rack11-host18%2Fsysinfo%2Fvar%2Fspool%2Fcrash%2Fchrome.20190115.132438.6330.dmp.txt?qk=AD5uMEsIMLjcwZZP4kNdt4rRF_YQkA8qawwnDonyujcgCzz3RlyqX_avoK3wuk-MEo85PXjOTimSQElfg_NR6tHjndXDaAeZ3DL5RozmoZtsr2CnBNi8mghE6l7sD1tKCyYOERx41W-zFLMV2mtT2J_VC0xKEK9wXVqTULfI6m_kYAfm9Ve0nDUT9EbNAvVaUIcTaQ8YqXLhAgT3esVCB1PF-rqVYyzJjRVCzcy5vxfG5YA-5ngC0RPCFA5Il6sYxwNJKyYhAI-j71rqy4yPc1Nu_XUg_J94YCqSWsiYeiP1xRHXn5lMWjyvndXGFnl_MRwGNITM4WJcwp9I5YlSZJqCcS1rJKcY7AQXdBsI65gq9XBCFyzPnoMZjBExmxwOdtkXkiHcuJ-ztxK-pM_ik7jGR4C2jh9V_llA87X6bS1-jvNTkkzrQHhtUt630DWkb3PPZQEOt90ThFeL7C6fNwPtl_y3TUdkroy98yLSUkA54msgCAQiYlvUevGUs4L7tODtvyKDaux7PTTyiFRQn4wlUAJcAjP9SGC1ngGyMSO1PxygWb-Gkyoj-tjdu7vAnCVHg3GuIg8LJRsr1539ujXLus5RFPb9ByWgQlnBzbISWn_ad1e8V8dHzOq1CeRMBAqAiTTbzpPVy4mUbEgLuD4lz_nm1cCQ-ptOIMkiIBOOLZzInzknHM4yEWmrSxHlD9YZlOsvbbGHdJ-88pqBkt7-qySeQa7pw6yn25ACU2It0O8sTdxVGT74n53cpGVg_JSbXmLchZaMlcK2haCBpeoBV_5K9BK_xEpO-ixuppZI90xERHkvMvaLPPI9KeBLTFV4CKF8zHoeAUmL7Fio6lML5lb04aAqMeR-WKDEj2aIqzpHTa0OyRyaH_qGjGXf-34SlXsm8e9aN4kFDSm4iftuGKyRcPicp1ulWv-JcJhB2_zP0cxgRqw



 
Cc: sonnyrao@chromium.org llozano@chromium.org
Thanks!

+sonnyrao, llozano, who are also looking into mlocking.

We need Chrome to spin up fully (incl. renderers getting into and staying in a reasonable state) in order for this test to pass. If Chrome is being really crashy on its own, that may cause this test to fail.

If Chrome is otherwise stable, maybe the test isn't properly starting Chrome up?
Hrm. Looks like it's been failing for peach_*, peach_pit, nyan_*, and daisy_* pretty consistently for as long as stainless is willing to go back.

According to https://www.chromium.org/chromium-os/developer-information-for-chrome-os-devices, all of those are ARM with kernels <= 3.10. Maybe mlock just doesn't work on those devices? Or maybe /proc has a different output format for mlocked memory on them?
I think this is the reason:


https://chromium-review.googlesource.com/c/chromiumos/overlays/board-overlays/+/441929

basically, some kernel's don't support huge pages.

can your test check if hugepages are enabled?



Ah, good catch. I'll look into that.

Are we still interested in locking this region of memory even if hugepages aren't enabled? (Sonny?)
hmm why don't those kernels support hugepages?  Is it that they don't support transparent hugepages maybe?

I think mlocking even without hugepages is okay.  I want to end up with something like this anyway when we get rid of min_filelist_kbytes.
We chatted offline, and are opting to disable this test for platforms which don't have hugepages enabled.

CL: https://chromium-review.googlesource.com/c/chromiumos/platform/tast-tests/+/1413873

Comment 7 by afakhry@chromium.org, Jan 17 (5 days ago)

Test is still failing: peach_pit-tot-chrome-pfq-informational: https://cros-goldeneye.corp.google.com/chromeos/healthmonitoring/buildDetails?buildbucketId=8924078828772450016

I can see that the CL in #6 hasn't gone through the CQ yet.
Project Member

Comment 8 by bugdroid1@chromium.org, Jan 18 (4 days ago)

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/overlays/chromiumos-overlay/+/f87a2f69e8aa0cc2a4c9d0cc5cc855231a845df6

commit f87a2f69e8aa0cc2a4c9d0cc5cc855231a845df6
Author: George Burgess IV <gbiv@google.com>
Date: Fri Jan 18 08:45:54 2019

tast-use-flags: add `transparent_hugepage` to IUSE

This is needed so that we can selectively disable our mlock test

BUG=chromium:922254
TEST=emerged/ran on daisy (disabled test) and falco (enabled test)

Change-Id: Iece31f1c5cc4eab03081a3273f6689a1a22179d3
Reviewed-on: https://chromium-review.googlesource.com/1415713
Commit-Ready: George Burgess <gbiv@chromium.org>
Tested-by: George Burgess <gbiv@chromium.org>
Reviewed-by: Dan Erat <derat@chromium.org>
Reviewed-by: Shuhei Takahashi <nya@chromium.org>

[rename] https://crrev.com/f87a2f69e8aa0cc2a4c9d0cc5cc855231a845df6/chromeos-base/tast-use-flags/tast-use-flags-0.0.1-r21.ebuild
[modify] https://crrev.com/f87a2f69e8aa0cc2a4c9d0cc5cc855231a845df6/chromeos-base/tast-use-flags/tast-use-flags-0.0.1.ebuild

Project Member

Comment 9 by bugdroid1@chromium.org, Jan 19 (3 days ago)

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/platform/tast-tests/+/408c6b5d3c5d057be81393ae14352a48ed0e43ed

commit 408c6b5d3c5d057be81393ae14352a48ed0e43ed
Author: George Burgess IV <gbiv@google.com>
Date: Sat Jan 19 08:13:43 2019

platform: disable the mlock tast on some platforms

Depending on the set of flags used to build a particular Chrome OS
image, it may/may not be possible to map things to hugepages. We should
skip platforms that don't have them enabled.

Corresponding feature addition:
I70ce57f01fb981c2937a59fc0b053d13e98635a2

BUG=chromium:922254
TEST=ran on daisy (disabled test) and falco (enabled test)
CQ-DEPEND=CL:1415684, CL:1415713

Change-Id: If7315194ee69a4c7219ace859849289e5655fcd1
Reviewed-on: https://chromium-review.googlesource.com/1413873
Commit-Ready: ChromeOS CL Exonerator Bot <chromiumos-cl-exonerator@appspot.gserviceaccount.com>
Tested-by: George Burgess <gbiv@chromium.org>
Reviewed-by: Dan Erat <derat@chromium.org>
Reviewed-by: Shuhei Takahashi <nya@chromium.org>

[modify] https://crrev.com/408c6b5d3c5d057be81393ae14352a48ed0e43ed/src/chromiumos/tast/local/bundles/cros/platform/chrome_mlocked.go

Project Member

Comment 10 by bugdroid1@chromium.org, Jan 19 (3 days ago)

The following revision refers to this bug:
  https://chromium.googlesource.com/chromiumos/platform/tast/+/28e8fc58d16bf69a83f88b291cc8923d0f8b2ee0

commit 28e8fc58d16bf69a83f88b291cc8923d0f8b2ee0
Author: George Burgess IV <gbiv@google.com>
Date: Sat Jan 19 08:13:43 2019

local_test_runner: add the transparent_hugepage feature

Complements the IUSE addition here:
Iece31f1c5cc4eab03081a3273f6689a1a22179d3

BUG=chromium:922254
TEST=emerged/ran on daisy (disabled test) and falco (enabled test)
CQ-DEPEND=CL:1415713
Change-Id: I70ce57f01fb981c2937a59fc0b053d13e98635a2
Reviewed-on: https://chromium-review.googlesource.com/1415684
Commit-Ready: ChromeOS CL Exonerator Bot <chromiumos-cl-exonerator@appspot.gserviceaccount.com>
Tested-by: George Burgess <gbiv@chromium.org>
Reviewed-by: George Burgess <gbiv@chromium.org>

[modify] https://crrev.com/28e8fc58d16bf69a83f88b291cc8923d0f8b2ee0/src/chromiumos/cmd/local_test_runner/main.go
[modify] https://crrev.com/28e8fc58d16bf69a83f88b291cc8923d0f8b2ee0/docs/test_dependencies.md

Sign in to add a comment