New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 631022 link

Starred by 1 user

Issue metadata

Status: Fixed
Owner:
Closed: Jul 2016
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: Android
Pri: 1
Type: Bug

Blocking:
issue 589726



Sign in to add a comment

Android Nexus7v2 Perf (1) extremely flaky System Health tests

Project Member Reported by picksi@chromium.org, Jul 25 2016

Issue description

https://uberchromegw.corp.google.com/i/chromium.perf/builders/Android%20Nexus7v2%20Perf%20%281%29?numbuilds=50

Bot appears to be extremely flaky (75% red), system health tests are failing:

system_health.memory_mobile failed page_cycler.typical_25
system_health.memory_mobile failed system_health.memory_mobile.reference

adding petr to check if there is an underlying cause.
 
Blocking: 589726
Labels: Hotlist-SystemHealth
Owner: u...@chromium.org
Status: Assigned (was: Untriaged)
I've gone through the five most recent failures:

https://build.chromium.org/p/chromium.perf/builders/Android%20Nexus7v2%20Perf%20%281%29/builds/3258/steps/system_health.memory_mobile/logs/stdio
https://build.chromium.org/p/chromium.perf/builders/Android%20Nexus7v2%20Perf%20%281%29/builds/3257/steps/system_health.memory_mobile/logs/stdio
https://build.chromium.org/p/chromium.perf/builders/Android%20Nexus7v2%20Perf%20%281%29/builds/3256/steps/system_health.memory_mobile/logs/stdio
https://build.chromium.org/p/chromium.perf/builders/Android%20Nexus7v2%20Perf%20%281%29/builds/3255/steps/system_health.memory_mobile/logs/stdio
https://build.chromium.org/p/chromium.perf/builders/Android%20Nexus7v2%20Perf%20%281%29/builds/3253/steps/system_health.memory_mobile/logs/stdio

And all of them have a common denominator: browse:social:facebook

Traceback (most recent call last):
  File "/b/build/slave/Android_Nexus7v2_Perf__1_/build/src/third_party/catapult/telemetry/telemetry/internal/story_runner.py", line 85, in _RunStoryAndProcessErrorIfNeeded
    state.RunStory(results)
  File "/b/build/slave/Android_Nexus7v2_Perf__1_/build/src/third_party/catapult/telemetry/telemetry/page/shared_page_state.py", line 319, in RunStory
    self._current_page.Run(self)
  File "/b/build/slave/Android_Nexus7v2_Perf__1_/build/src/third_party/catapult/telemetry/telemetry/page/__init__.py", line 99, in Run
    self.RunPageInteractions(action_runner)
  File "/b/build/slave/Android_Nexus7v2_Perf__1_/build/src/tools/perf/page_sets/system_health/system_health_story.py", line 79, in RunPageInteractions
    self._DidLoadDocument(action_runner)
  File "/b/build/slave/Android_Nexus7v2_Perf__1_/build/src/tools/perf/page_sets/system_health/browsing_stories.py", line 60, in _DidLoadDocument
    self._NavigateToItem(action_runner, i)
  File "/b/build/slave/Android_Nexus7v2_Perf__1_/build/src/tools/perf/page_sets/system_health/browsing_stories.py", line 29, in _NavigateToItem
    self._ClickLink(action_runner, item_selector)
  File "/b/build/slave/Android_Nexus7v2_Perf__1_/build/src/tools/perf/page_sets/system_health/browsing_stories.py", line 32, in _ClickLink
    action_runner.WaitForElement(element_function=element_function)
  File "/b/build/slave/Android_Nexus7v2_Perf__1_/build/src/third_party/catapult/telemetry/telemetry/internal/actions/action_runner.py", line 206, in WaitForElement
    timeout_in_seconds=timeout_in_seconds))
  File "/b/build/slave/Android_Nexus7v2_Perf__1_/build/src/third_party/catapult/telemetry/telemetry/internal/actions/action_runner.py", line 45, in _RunAction
    action.RunAction(self._tab)
  File "/b/build/slave/Android_Nexus7v2_Perf__1_/build/src/third_party/catapult/telemetry/telemetry/internal/actions/wait.py", line 22, in RunAction
    wait=True, timeout_in_seconds=self.timeout_in_seconds)
  File "/b/build/slave/Android_Nexus7v2_Perf__1_/build/src/third_party/catapult/telemetry/telemetry/internal/actions/page_action.py", line 113, in EvaluateCallbackWithElement
    tab.WaitForJavaScriptExpression(code, timeout_in_seconds)
  File "/b/build/slave/Android_Nexus7v2_Perf__1_/build/src/third_party/catapult/telemetry/telemetry/internal/browser/web_contents.py", line 136, in WaitForJavaScriptExpression
    e.message + '\n' + debug_message)
TimeoutException: Timed out while waiting 60s for IsJavaScriptExpressionTrue.
Console output:

Here's the screenshot: https://console.developers.google.com/m/cloudstorage/b/chrome-telemetry-output/o/profiler-file-id_0-2016-07-25_03-49-0952571.png

Ulan: Could you please have a look? I can see that the screenshot is at the very bottom of the page (you can't scroll further down). Is there any chance that this is causing the problem (the link that the story wants to click on doesn't show unless you click "See all")?

Comment 2 by u...@chromium.org, Jul 25 2016

FB fetches posts dynamically during scrolling. The screenshot is very close to the beginning of the page. The actual posts come afterwards.

So it looks like on Nexus7v2 we are not scrolling far enough so the fetching logic does not trigger.

If that is true, then setting MAIN_PAGE_SCROLL_REPEAT = 1 (or more) for FacebookMobileStory should fix it.

I'll check it.
Project Member

Comment 3 by bugdroid1@chromium.org, Jul 26 2016

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/aa939aefe4a8f0c33705ed006bcab8bf13134820

commit aa939aefe4a8f0c33705ed006bcab8bf13134820
Author: ulan <ulan@chromium.org>
Date: Tue Jul 26 10:06:43 2016

[system-health] Fix mobile facebook story for Nexus7v2.

Facebook fetches items on demand while scrolling. Before this patch
Nexus7v2 was timing out while waiting for items because the scroll
distance was not enough to trigger fetching of items.

This patch increases the scroll distance and updates WPR.

*** NOTE TO SHERRIF ***
Regressions are expected in browse:social:facebook story.

BUG= 631022 

Review-Url: https://codereview.chromium.org/2175383003
Cr-Commit-Position: refs/heads/master@{#407756}

[modify] https://crrev.com/aa939aefe4a8f0c33705ed006bcab8bf13134820/tools/perf/page_sets/data/system_health_mobile.json
[add] https://crrev.com/aa939aefe4a8f0c33705ed006bcab8bf13134820/tools/perf/page_sets/data/system_health_mobile_028.wpr.sha1
[modify] https://crrev.com/aa939aefe4a8f0c33705ed006bcab8bf13134820/tools/perf/page_sets/system_health/browsing_stories.py

Comment 4 by picksi@chromium.org, Jul 26 2016

The change in #3 landed in build #3265
(https://uberchromegw.corp.google.com/i/chromium.perf/builders/Android%20Nexus7v2%20Perf%20%281%29/builds/3265)

System Health was green for this run (great! Thanks Ulan).

The bot has remained red due to failed page_cycler.typical_25. This test has been flaky on previous occasions, hopefully #3266 will be green...

https://uberchromegw.corp.google.com/i/chromium.perf/builders/Android%20Nexus7v2%20Perf%20%281%29/builds/3266


Comment 5 by u...@chromium.org, Jul 26 2016

Status: Fixed (was: Assigned)
Thank you for update, picksi@.

3266 is green, closing the bug :)

Sign in to add a comment