New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 860366 link

Starred by 1 user

Issue metadata

Status: Assigned
Owner:
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: Chrome
Pri: 2
Type: Bug-Regression

Blocked on:
issue 868755



Sign in to add a comment

5 pages of page_cycler_v2.typical_25 would fail using local build ChromiumOS image

Reported by lingyun....@intel.com, Jul 5

Issue description

UserAgent: Mozilla/5.0 (Windows NT 6.3; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/67.0.3396.99 Safari/537.36

Steps to reproduce the problem:
(1)sync Chromium OS source code with manifest 10798.0.0-rc2.xml from paladin folder in manifest-versions repo (https://chromium.googlesource.com/chromiumos/manifest-versions)
(2)sync Chromium to 69.0.3464.0
(3)build local image with below commands:
	1) cros_sdk --enter --chrome_root=chrome_root
	2) export CHROME_ORIGIN=LOCAL_SOURCE
	3) export BOARD=reef
	4) ./setup_board --board=$BOARD
	5) cros_workon --board=$BOARD start chromeos-chrome
	6) USE="afdo_use thinlto" ./build_packages --board=$BOARD
	7) ./build_image --board=$BOARD --noenable_rootfs_verification test
(4) run page_cycler_v2.typical_25 using local build ChromiumOS image and check the results

What is the expected behavior?
All pages are loaded successfully with metric values using local build ChromiumOS images

What went wrong?
5 pages would fail:
(1) economist
(2) allrecipes
(3) money.cnn
(4) premierleague
(5) ign
However, no failure observed if using partner ChromeOS image with the same version (i.e., 10798.0.0).

The error messages from local build ChromiumOS image are shown as:
Traceback (most recent call last):
  File "/home/test/chromium_telemetry_new_5/src/third_party/catapult/telemetry/telemetry/internal/story_runner.py", line 123, in _RunStoryAndProcessErrorIfNeeded
    state.RunStory(results)
  File "/home/test/chromium_telemetry_new_5/src/third_party/catapult/common/py_trace_event/py_trace_event/trace_event_impl/decorators.py", line 75, in traced_function
    return func(*args, **kwargs)
  File "/home/test/chromium_telemetry_new_5/src/third_party/catapult/telemetry/telemetry/page/shared_page_state.py", line 311, in RunStory
    self._current_page.Run(self)
  File "/home/test/chromium_telemetry_new_5/src/third_party/catapult/telemetry/telemetry/page/__init__.py", line 100, in Run
    self.RunPageInteractions(action_runner)
  File "/home/test/chromium_telemetry_new_5/src/tools/perf/page_sets/typical_25.py", line 27, in RunPageInteractions
    'performance.timing.loadEventStart > 0')
  File "/home/test/chromium_telemetry_new_5/src/third_party/catapult/common/py_trace_event/py_trace_event/trace_event_impl/decorators.py", line 75, in traced_function
    return func(*args, **kwargs)
  File "/home/test/chromium_telemetry_new_5/src/third_party/catapult/telemetry/telemetry/internal/actions/action_runner.py", line 261, in WaitForJavaScriptCondition
    return self._tab.WaitForJavaScriptCondition(*args, **kwargs)
  File "/home/test/chromium_telemetry_new_5/src/third_party/catapult/common/py_trace_event/py_trace_event/trace_event_impl/decorators.py", line 75, in traced_function
    return func(*args, **kwargs)
  File "/home/test/chromium_telemetry_new_5/src/third_party/catapult/telemetry/telemetry/internal/browser/web_contents.py", line 239, in WaitForJavaScriptCondition
    return self._inspector_backend.WaitForJavaScriptCondition(*args, **kwargs)
  File "/home/test/chromium_telemetry_new_5/src/third_party/catapult/common/py_trace_event/py_trace_event/trace_event_impl/decorators.py", line 75, in traced_function
    return func(*args, **kwargs)
  File "/home/test/chromium_telemetry_new_5/src/third_party/catapult/telemetry/telemetry/internal/backends/chrome_inspector/inspector_backend.py", line 302, in WaitForJavaScriptCondition
    e.message + '\n' + debug_message)
TimeoutException: Timed out while waiting 60s for IsJavaScriptExpressionTrue.

Did this work before? Yes 

Chrome version: 69.0.3464.0  Channel: n/a
OS Version: 10798.0.0
Flash Version:
 
Cc: hong.zh...@intel.com
Cc: laszio@chromium.org
Components: Tools>ChromeOS-Toolchain
Cc: ksakamoto@chromium.org kouhei@chromium.org
Components: -Tools>ChromeOS-Toolchain Speed>Telemetry
Owner: kouhei@chromium.org
Hi Kouhei, according to "Chrome Benchmark Reset", the owner of page_cycler series is you and kasakamoto. Would you mind to take a look?
Hi kouhei@ and ksakamoto@, would you mind to take a look and share your comments? 
Please let me know if any data needed, I could help to provide.
Sorry, I don't have much insight now, since we retired page_cycler_v2 but for ChromeOS.
Cc: cywang@chromium.org

Comment 7 Deleted

+Chrome OS Perf team lead. Perhaps we should move to loading.desktop.
Cc: vovoy@chromium.org
Please provide the full command of (4) run page_cycler_v2.typical_25, and attach the full test log. Thanks.
Full command of (4) is :
./run_benchmark --browser=cros-chrome --remote=$DUT_IP page_cycler_v2.typical_25 --pageset-repeat=1 --reset-results -v

Test log from telemetry is also attached. I just ran one page 'ign.com' which could reproduce the issue.
test-log-10798-cros
25.5 KB View Download
Hi vovoy@, I was wondering if you have any update on this issue?
Please let me know if any more data are needed.
Blockedon: 868755
page_cycler_v2 is deprecated, I will migrate it to loading.desktop on Chrome OS.
Hi lingyun-cai@, you can use the following benchmark for page loading time measuring.

./run_benchmark --browser=cros-chrome --remote=$DUT_IP --story-tag-filter=typical loading.desktop
Hi vovoy@, thanks for the update. We've tried loading.desktop on Chrome OS and it worked.

We noticed that there are multiple metrics such as FP/FCP/FMP and timeToOnload, which one do you pay more attention to on regular perf tracking?
timeToFirstContentfulPaint: Time until something nontrivial (e.g. text or image) is painted for the first time.
timeToFirstMeaningfulPaint: Time until the first paint where the page's primary content is visible (heuristically determined).
FirstPaint: Time until anything is painted for the current document.
timeToOnload: Time until the onload event. Because this has low correlation with user-perceived load time, we do not recommend using this as a key metric.

I would pay more attention on timeToFirstContentfulPaint.
We recently found the possible cause of the failures on page_cycler_v2 using local ChromiumOS image, which is related to site isolation.

On local build ChromiumOS, site isolation is enabled by default, while disabled by default on official ChromeOS. If we ran page_cycler_v2.typical_25 with Telemetry flag '--extra-browser-args="--disable-site-isolation-trials"' on local ChromiumOS, the failure pages would be loaded successfully.
Status: Assigned (was: Unconfirmed)
This issue has an owner, a component and a priority, but is still listed as untriaged or unconfirmed. By definition, this bug is triaged. Changing status to "assigned". Please reach out to me if you disagree with how I've done this.

Comment 20 by benhenry@google.com, Jan 16 (6 days ago)

Components: Test>Telemetry

Comment 21 by benhenry@google.com, Jan 16 (6 days ago)

Components: -Speed>Telemetry

Sign in to add a comment