startup.large_profile.warm.blank_page on (102b) GPU on Windows on Windows-2008ServerR2-SP1 failing on 2 builders |
|||||||
Issue descriptionstartup.large_profile.warm.blank_page on (102b) GPU on Windows on Windows-2008ServerR2-SP1 failing on 2 builders Builders failed on: - Win 7 Perf: https://build.chromium.org/p/chromium.perf/builders/Win%207%20Perf - Win 7 x64 Perf: https://build.chromium.org/p/chromium.perf/builders/Win%207%20x64%20Perf Passing at r458114 Failing at r458115:r458269 Could be related to https://codereview.chromium.org/2761063002 https://luci-logdog.appspot.com/v/?s=chrome%2Fbb%2Fchromium.perf%2FWin_7_Perf%2F507%2F%2B%2Frecipes%2Fsteps%2Fstartup.large_profile.warm.blank_page_on__102b__GPU_on_Windows_on_Windows-2008ServerR2-SP1%2F0%2Fstdout Traceback (most recent call last): File "c:\b\s\w\irr8855y\third_party\catapult\telemetry\telemetry\internal\story_runner.py", line 81, in _RunStoryAndProcessErrorIfNeeded state.WillRunStory(story) File "c:\b\s\w\irr8855y\third_party\catapult\common\py_trace_event\py_trace_event\trace_event_impl\decorators.py", line 52, in traced_function return func(*args, **kwargs) File "c:\b\s\w\irr8855y\tools\perf\page_sets\pregenerated_profile_shared_state.py", line 30, in WillRunStory super(PregeneratedProfileSharedState, self).WillRunStory(page) File "c:\b\s\w\irr8855y\third_party\catapult\common\py_trace_event\py_trace_event\trace_event_impl\decorators.py", line 52, in traced_function return func(*args, **kwargs) File "c:\b\s\w\irr8855y\third_party\catapult\telemetry\telemetry\page\shared_page_state.py", line 214, in WillRunStory self._StartBrowser(page) File "c:\b\s\w\irr8855y\third_party\catapult\common\py_trace_event\py_trace_event\trace_event_impl\decorators.py", line 52, in traced_function return func(*args, **kwargs) File "c:\b\s\w\irr8855y\third_party\catapult\telemetry\telemetry\page\shared_page_state.py", line 174, in _StartBrowser self._browser = self._possible_browser.Create(self._finder_options) File "c:\b\s\w\irr8855y\third_party\catapult\telemetry\telemetry\internal\backends\chrome\desktop_browser_finder.py", line 66, in Create self._flash_path, self._is_content_shell, self._browser_directory) File "c:\b\s\w\irr8855y\third_party\catapult\common\py_trace_event\py_trace_event\trace_event_impl\decorators.py", line 52, in traced_function return func(*args, **kwargs) File "c:\b\s\w\irr8855y\third_party\catapult\telemetry\telemetry\internal\backends\chrome\desktop_browser_backend.py", line 143, in __init__ self._SetupProfile() File "c:\b\s\w\irr8855y\third_party\catapult\common\py_trace_event\py_trace_event\trace_event_impl\decorators.py", line 52, in traced_function return func(*args, **kwargs) File "c:\b\s\w\irr8855y\third_party\catapult\telemetry\telemetry\internal\backends\chrome\desktop_browser_backend.py", line 175, in _SetupProfile shutil.copytree(profile_dir, self._tmp_profile_dir) File "c:\b\depot_tools\python276_bin\lib\shutil.py", line 208, in copytree raise Error, errors Error: [('c:\\b\\s\\w\\itu7xdj9\\tmphgrwm4\\BrowserMetrics-active.pma', 'c:\\b\\s\\w\\itu7xdj9\\tmpdddtsm\\BrowserMetrics-active.pma', "[Errno 13] Permission denied: 'c:\\\\b\\\\s\\\\w\\\\itu7xdj9\\\\tmphgrwm4\\\\BrowserMetrics-active.pma'"), ('c:\\b\\s\\w\\itu7xdj9\\tmphgrwm4\\Default\\Cache\\data_0', 'c:\\b\\s\\w\\itu7xdj9\\tmpdddtsm\\Default\\Cache\\data_0', "[Errno 13] Permission denied: 'c:\\\\b\\\\s\\\\w\\\\itu7xdj9\\\\tmphgrwm4\\\\Default\\\\Cache\\\\data_0'"), ('c:\\b\\s\\w\\itu7xdj9\\tmphgrwm4\\Default\\Cache\\data_1', 'c:\\b\\s\\w\\itu7xdj9\\tmpdddtsm\\Default\\Cache\\data_1', "[Errno 13] Permission denied: 'c:\\\\b\\\\s\\\\w\\\\itu7xdj9\\\\tmphgrwm4\\\\Default\\\\Cache\\\\data_1'"), ('c:\\b\\s\\w\\itu7xdj9\\tmphgrwm4\\Default\\Cache\\data_2', 'c:\\b\\s\\w\\itu7xdj9\\tmpdddtsm\\Default\\Cache\\data_2', "[Errno 13] Permission denied: 'c:\\\\b\\\\s\\\\w\\\\itu7xdj9\\\\tmphgrwm4\\\\Default\\\\Cache\\\\data_2'"), ('c:\\b\\s\\w\\itu7xdj9\\tmphgrwm4\\Default\\Cache\\data_3', 'c:\\b\\s\\w\\itu7xdj9\\tmpdddtsm\\Default\\Cache\\data_3', "[Errno 13] Permission denied: 'c:\\\\b\\\\s\\\\w\\\\itu7xdj9\\\\tmphgrwm4\\\\Default\\\\Cache\\\\data_3'"), ('c:\\b\\s\\w\\itu7xdj9\\tmphgrwm4\\lockfile', 'c:\\b\\s\\w\\itu7xdj9\\tmpdddtsm\\lockfile', "[Errno 2] No such file or directory: 'c:\\\\b\\\\s\\\\w\\\\itu7xdj9\\\\tmphgrwm4\\\\lockfile'")]
,
Mar 22 2017
,
Mar 22 2017
=== Auto-CCing suspected CL author gangwu@chromium.org === Hi gangwu@chromium.org, the bisect results pointed to your CL, please take a look at the results. === BISECT JOB RESULTS === Test failure found with culprit Suspected Commit Author : gangwu Commit : e8cbd06453b8b4dfc01fa61d004fa0469f42bd98 Date : Mon Mar 20 23:24:43 2017 Subject: [sync] Add typed url sync metadata to the history db Bisect Details Configuration: win_perf_bisect Benchmark : startup.large_profile.warm.blank_page Metric : messageloop_start_time/messageloop_start_time Revision Exit Code N chromium@458114 0 +- N/A 20 good chromium@458192 0 +- N/A 20 good chromium@458231 0 +- N/A 20 good chromium@458232 1 +- N/A 20 bad <-- chromium@458233 1 +- N/A 20 bad chromium@458234 1 +- N/A 20 bad chromium@458236 1 +- N/A 20 bad chromium@458241 1 +- N/A 20 bad chromium@458250 1 +- N/A 20 bad chromium@458269 1 +- N/A 20 bad To Run This Test src/tools/perf/run_benchmark -v --browser=release --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests startup.large_profile.warm.blank_page Debug Info https://chromeperf.appspot.com/buildbucket_job_status/8984408829533675584 Is this bisect wrong? https://chromeperf.appspot.com/bad_bisect?try_job_id=6452657289428992 | O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq | X | for more information addressing perf regression bugs. For feedback, | / \ | file a bug with component Speed>Bisection. Thank you!
,
Mar 22 2017
ashleymarie@chromium.org, can you start another bisect job? I don't feel it related to my CL.
,
Mar 22 2017
Started bisect job https://chromeperf.appspot.com/buildbucket_job_status/8984390345883469744
,
Mar 22 2017
I kicked off a second bisect, but it does look like it could be related, the test started failing due to an error in copying the profile when your CL adds to the history db.
,
Mar 22 2017
oh, yes, my CL will updating an sqlite table during startup, if the table is really large, then it could take a long time. but that should only happen once for each existing profile. So is there a way to fix it, what is the profile looks like, can I update profile? somehow, I cannot see the log for some of steps in https://build.chromium.org/p/tryserver.chromium.perf/builders/win_perf_bisect/builds/7222. I hope I can get detail about the tests.
,
Mar 22 2017
+Ned can you help with next steps on debugging?
,
Mar 22 2017
+Erik: can you have gangwu@ with the profile failure?
,
Mar 23 2017
=== BISECT JOB RESULTS === Test failure found with culprit Suspected Commit Author : gangwu Commit : e8cbd06453b8b4dfc01fa61d004fa0469f42bd98 Date : Mon Mar 20 23:24:43 2017 Subject: [sync] Add typed url sync metadata to the history db Bisect Details Configuration: win_perf_bisect Benchmark : startup.large_profile.warm.blank_page Metric : messageloop_start_time/messageloop_start_time Revision Exit Code N chromium@458114 0 +- N/A 20 good chromium@458192 0 +- N/A 20 good chromium@458231 0 +- N/A 20 good chromium@458232 1 +- N/A 20 bad <-- chromium@458233 1 +- N/A 20 bad chromium@458234 1 +- N/A 20 bad chromium@458236 1 +- N/A 20 bad chromium@458241 1 +- N/A 20 bad chromium@458250 1 +- N/A 20 bad chromium@458269 1 +- N/A 20 bad To Run This Test src/tools/perf/run_benchmark -v --browser=release --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests startup.large_profile.warm.blank_page Debug Info https://chromeperf.appspot.com/buildbucket_job_status/8984390345883469744 Is this bisect wrong? https://chromeperf.appspot.com/bad_bisect?try_job_id=5798935886561280 | O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq | X | for more information addressing perf regression bugs. For feedback, | / \ | file a bug with component Speed>Bisection. Thank you!
,
Mar 23 2017
"""
oh, yes, my CL will updating an sqlite table during startup, if the table is really large, then it could take a long time. but that should only happen once for each existing profile.
So is there a way to fix it, what is the profile looks like, can I update profile?
"""
The large profile is based on my real profile from a couple of years ago. If a migration is causing a timeout, then that suggests that the migration might be taking longer than you think. Do you have metrics/measurements for how long the migration takes?
The large profiles are available from telemetry at:
chromium src/tools/perf/generated_profiles/darwin/large_profile.zip.sha1
1: d40189a1f471213d06314aaa2095bc35c1027f0c ...
chromium src/tools/perf/generated_profiles/win32/large_profile.zip.sha1
1: d7d2b8e928e63079e4f1e97e9d8a36518cc4a443 ...
chromium src/tools/perf/generated_profiles/linux2/large_profile.zip.sha1
1: 00afe3f1eb2624db9ff71bdf99f589e8383be2d0 ...
nednguyen: I forget off the top of my head the name of the system that telemetry uses to download binaries from sha1 hashes.
,
Mar 23 2017
I see two possibilities: 1) large_profile is very out-dated, and the migration is a pathological case that takes too long. We need to update the large_profile. 2) large_profile is very out-dated, but the migration is not a pathological case, and demonstrates a real performance problem.
,
Mar 23 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/2afcd73c4d28db6974d3166cfe9427106f46d051 commit 2afcd73c4d28db6974d3166cfe9427106f46d051 Author: ashleymarie <ashleymarie@chromium.org> Date: Thu Mar 23 16:35:26 2017 Disabling startup.large_profile.warm.blank_page on Windows BUG= chromium:704137 Review-Url: https://codereview.chromium.org/2770913004 Cr-Commit-Position: refs/heads/master@{#459113} [modify] https://crrev.com/2afcd73c4d28db6974d3166cfe9427106f46d051/tools/perf/benchmarks/startup.py
,
Jan 17 2018
the test got deleted, so close this bug. https://chromium-review.googlesource.com/c/chromium/src/+/644107 |
|||||||
►
Sign in to add a comment |
|||||||
Comment 1 by 42576172...@developer.gserviceaccount.com
, Mar 22 2017