angle_perftests on Intel GPU on Windows on Windows-2008ServerR2-SP1 failing on chromium.perf/Win 7 Intel GPU Perf |
|||||||||||||
Issue descriptionangle_perftests on Intel GPU on Windows on Windows-2008ServerR2-SP1 failing on chromium.perf/Win 7 Intel GPU Perf Builders failed on: - Win 7 Intel GPU Perf: https://build.chromium.org/p/chromium.perf/builders/Win%207%20Intel%20GPU%20Perf Success at r455054 Failed at r455056:r455132 https://luci-logdog.appspot.com/v/?s=chrome%2Fbb%2Fchromium.perf%2FWin_7_Intel_GPU_Perf%2F490%2F%2B%2Frecipes%2Fsteps%2Fangle_perftests_on_Intel_GPU_on_Windows_on_Windows-2008ServerR2-SP1%2F0%2Fstdout Backtrace: RegisterProcTableCallback [0x000007FEF23F3ADA+775738] (No symbol) [0x000007FEF23350EC] (No symbol) [0x000007FEF233497A] DrvCreateLayerContext [0x000007FEF286FBCC+380] wglSwapMultipleBuffers [0x000007FEF12182F0+6304] wglCreateLayerContext [0x000007FEF1218B75+485] rx::DisplayWGL::initialize [0x000000013FED4C7E+622] egl::Display::initialize [0x000000013FE84EDF+431] egl::Initialize [0x000000013FE021B1+81] EGLWindow::initializeGL [0x000000013FDF50B9+473] ANGLERenderTest::SetUp [0x000000013FD8A5ED+269] testing::internal::HandleExceptionsInMethodIfSupported<testing::TestCase,void> [0x000000013FDE0EED+61] testing::Test::Run [0x000000013FDEB25A+74] testing::TestInfo::Run [0x000000013FDEB481+161] testing::TestCase::Run [0x000000013FDEB365+165] testing::internal::UnitTestImpl::RunAllTests [0x000000013FDEB832+562] testing::internal::HandleExceptionsInMethodIfSupported<testing::internal::UnitTestImpl,bool> [0x000000013FDE0F8D+61] testing::UnitTest::Run [0x000000013FDEB5C9+217] base::TestSuite::Run [0x000000013FDCD729+137] base::internal::Invoker<base::internal::BindState<int (__cdecl*)(base::TestSuite * __ptr64),base::internal::UnretainedWrapper<base::TestSuite> >,int __cdecl(void)>::Run [0x000000013FDC967F+63] base::UnitTestLauncherDelegate::GetTests [0x000000013FDCB030+944] base::LaunchUnitTestsSerially [0x000000013FDCB0C3+99] main [0x000000013FDC9709+113] __scrt_common_main_seh [0x00000001400D1ED9+285] (f:\dd\vctools\crt\vcstartup\src\startup\exe_common.inl:253) BaseThreadInitThunk [0x00000000776559ED+13] RtlUserThreadStart [0x000000007778C541+33] Running ['.\\angle_perftests.exe', '--test-launcher-print-test-stdio=always', '--test-launcher-jobs=1'] in None (env: {'TMP': 'C:\\Users\\CHROME~1\\AppData\\Local\\Temp', 'LC_NUMERIC': 'English_United States.UTF-8', 'COMPUTERNAME': 'BUILD166-M1', 'VS110COMNTOOLS': 'C:\\Program Files (x86)\\Microsoft Visual Studio 11.0\\Common7\\Tools\\', 'USERDOMAIN': 'GOLO', 'LC_CTYPE': 'English_United States.UTF-8', 'PSMODULEPATH': 'C:\\Windows\\system32\\WindowsPowerShell\\v1.0\\Modules\\', 'COMMONPROGRAMFILES': 'C:\\Program Files (x86)\\Common Files', 'PROCESSOR_IDENTIFIER': 'Intel64 Family 6 Model 60 Stepping 3, GenuineIntel', 'PROGRAMFILES': 'C:\\Program Files (x86)', 'PROCESSOR_REVISION': '3c03', 'HOME': 'c:\\Users\\chrome-bot', 'BOTO_CONFIG': 'c:\\Users\\chrome-bot\\.boto', 'PATH': 'c:\\b\\depot_tools\\python276_bin;c:\\b\\depot_tools\\python276_bin\\Scripts;C:\\Windows\\system32;C:\\Windows;C:\\Windows\\System32\\Wbem;C:\\Windows\\System32\\WindowsPowerShell\\v1.0\\;c:\\tools;c:\\b\\depot_tools;C:\\Program Files\\Microsoft SQL Server\\110\\Tools\\Binn\\;C:\\Program Files (x86)\\Windows Kits\\8.1\\Windows Performance Toolkit\\;C:\\Program Files (x86)\\Microsoft SDKs\\TypeScript\\1.0\\;C:\\Program Files (x86)\\QuickTime\\QTSystem\\;C:\\ProgramData\\chocolatey\\bin;C:\\Program Files\\Puppet Labs\\Puppet\\bin', 'SWARMING_BOT_ID': 'build166-m1', 'PROGRAMFILES(X86)': 'C:\\Program Files (x86)', 'COMSPEC': 'C:\\Windows\\system32\\cmd.exe', 'LANG': 'en_US.UTF-8', 'SWARMING_SERVER': 'https://chromium-swarm.appspot.com', 'TEMP': 'c:\\b\\s\\w\\itk22erl', 'LC_MONETARY': 'English_United States.UTF-8', 'CHROME_DEVEL_SANDBOX': '/opt/chromium/chrome_sandbox', 'COMMONPROGRAMFILES(X86)': 'C:\\Program Files (x86)\\Common Files', 'PROCESSOR_ARCHITECTURE': 'x86', 'CHOCOLATEYINSTALL': 'C:\\ProgramData\\chocolatey', 'PATHEXT': '.COM;.EXE;.BAT;.CMD;.VBS;.VBE;.JS;.JSE;.WSF;.WSH;.MSC', 'SWARMING_HEADLESS': '1', 'ALLUSERSPROFILE': 'C:\\ProgramData', 'LOCALAPPDATA': 'C:\\Users\\chrome-bot\\AppData\\Local', 'HOMEPATH': '\\Users\\chrome-bot', 'VS120COMNTOOLS': 'C:\\Program Files (x86)\\Microsoft Visual Studio 12.0\\Common7\\Tools\\', 'PROGRAMW6432': 'C:\\Program Files', 'USERNAME': 'chrome-bot', 'LOGONSERVER': '\\\\AD2', 'SYSTEMROOT': 'C:\\Windows', 'WINDOWS_TRACING_FLAGS': '3', 'PROGRAMDATA': 'C:\\ProgramData', 'SWARMING_TASK_ID': '34c30f127695e411', 'PROMPT': '$P$G', 'USERDNSDOMAIN': 'GOLO.CHROMIUM.ORG', 'NO_GCE_CHECK': 'False', 'SESSIONNAME': 'Console', 'LC_ALL': 'English_United States.UTF-8', 'LC_COLLATE': 'English_United States.UTF-8', 'FP_NO_HOST_CHECK': 'NO', 'WINDIR': 'C:\\Windows', 'WINDOWS_TRACING_LOGFILE': 'C:\\BVTBin\\Tests\\installpackage\\csilogfile.log', 'HOMEDRIVE': 'C:', 'SYSTEMDRIVE': 'C:', 'NUMBER_OF_PROCESSORS': '8', 'APPDATA': 'C:\\Users\\chrome-bot\\AppData\\Roaming', 'PROCESSOR_LEVEL': '6', 'PROCESSOR_ARCHITEW6432': 'AMD64', 'LC_TIME': 'English_United States.UTF-8', 'COMMONPROGRAMW6432': 'C:\\Program Files\\Common Files', 'OS': 'Windows_NT', 'PUBLIC': 'C:\\Users\\Public', 'USERPROFILE': 'C:\\Users\\chrome-bot'}) Command ['.\\angle_perftests.exe', '--test-launcher-print-test-stdio=always', '--test-launcher-jobs=1'] returned exit code 1
,
Mar 23 2017
,
Mar 23 2017
,
Mar 23 2017
,
Mar 23 2017
This issue is really weird, I'm trying to log into the bots so I can see what's up, and maybe try disabling the Matrox card or upgrading the drivers. It's the exact same crash as in issue 680708 so I suspect a driver-related problem.
,
Mar 24 2017
I'll be away next week, if I'm not able to fix this today.
,
Mar 28 2017
Started bisect job https://chromeperf.appspot.com/buildbucket_job_status/8983847485423808608
,
Mar 28 2017
=== BISECT JOB RESULTS === NO Test failure found Bisect Details Configuration: winx64intel_perf_bisect Benchmark : angle_perftests Metric : BlitFramebufferPerf_d3d11_color/score Revision Exit Code N chromium@455055 1 +- N/A 2 good chromium@455135 1 +- N/A 2 bad To Run This Test .\src\out\Release_x64\angle_perftests.exe --test-launcher-print-test-stdio=always --test-launcher-jobs=1 Debug Info https://chromeperf.appspot.com/buildbucket_job_status/8983847485423808608 Is this bisect wrong? https://chromeperf.appspot.com/bad_bisect?try_job_id=5222631604748288 | O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq | X | for more information addressing perf regression bugs. For feedback, | / \ | file a bug with component Speed>Bisection. Thank you!
,
May 4 2017
Issue 718294 has been merged into this issue.
,
May 4 2017
ashleymarie@chromium.org, do you know what happened to the test stdout? It used to display the error in the logs, now all stdio seems hidden.
,
May 4 2017
No idea hmm Maybe Ned would know?
,
May 4 2017
I think this is regression happens due to the process of migrating these tests to swarming. The angle_perftests.exe script is executed through run_gtest_perf_test.py, which seems to swallow the test stdout upon error: See: https://cs.chromium.org/chromium/src/testing/scripts/run_gtest_perf_test.py?rcl=66a2016758be5a47057b8580f9d0432b7de570f8&l=96 https://cs.chromium.org/chromium/src/testing/scripts/common.py?rcl=9cc81c66fbe4e06f84a5d8647faf1a2fab5ecf38&l=73 This is because normally, run_gtest_perf_test would rely on the stdout to parse test output to json format & upload to the perf dashboard. Upon test failure, it gets confused. The short term fix for this is to always print out content of |tempfile_path|. The long term fix is these benchmark harnesses should support s.t like "--json-output-path=<...>", and the run_gtest_perf_test should not try to mess around with the test stdout.
,
May 4 2017
,
May 4 2017
,
May 4 2017
Just figured this out - seems that Intel/Win7/D3D9 doesn't like it when our vertex shader doesn't have any vertex attributes. Will have a fix up today or tomorrow.
,
May 4 2017
Thanks for digging into this jmadill!
,
May 5 2017
The following revision refers to this bug: https://chromium.googlesource.com/angle/angle/+/06f76552c04dbe5600654bb09992954a12691b9d commit 06f76552c04dbe5600654bb09992954a12691b9d Author: Jamie Madill <jmadill@chromium.org> Date: Fri May 05 14:48:13 2017 UniformsPerf: Define a vertex attribute. This seems to fix a crash on D3D9/Win7/Intel. BUG= chromium:704211 Change-Id: If600a6b8409774503f28ca122d87ef723e4ea2e5 Reviewed-on: https://chromium-review.googlesource.com/497649 Reviewed-by: Geoff Lang <geofflang@chromium.org> Commit-Queue: Jamie Madill <jmadill@chromium.org> [modify] https://crrev.com/06f76552c04dbe5600654bb09992954a12691b9d/src/tests/perf_tests/UniformsPerf.cpp
,
May 5 2017
Fix will be rolled out Monday, since we don't roll on Fridays.
,
May 8 2017
The following revision refers to this bug: https://chromium.googlesource.com/chromium/src.git/+/e0f867b3e07d76f29db264c2bfa1c4d67dbc8e59 commit e0f867b3e07d76f29db264c2bfa1c4d67dbc8e59 Author: jmadill <jmadill@chromium.org> Date: Mon May 08 17:34:20 2017 Roll ANGLE aa7203e..b4cf565 https://chromium.googlesource.com/angle/angle.git/+log/aa7203e..b4cf565 BUG= chromium:704211 TBR=geofflang@chromium.org TEST=bots CQ_INCLUDE_TRYBOTS=master.tryserver.chromium.win:win_optional_gpu_tests_rel;master.tryserver.chromium.mac:mac_optional_gpu_tests_rel;master.tryserver.chromium.linux:linux_optional_gpu_tests_rel;master.tryserver.chromium.android:android_optional_gpu_tests_rel Review-Url: https://codereview.chromium.org/2868893002 Cr-Commit-Position: refs/heads/master@{#470036} [modify] https://crrev.com/e0f867b3e07d76f29db264c2bfa1c4d67dbc8e59/DEPS
,
May 9 2017
The first run with the ANGLE roll in in comment 19 has a working angle_perftests again: https://build.chromium.org/p/chromium.perf/builders/Win%207%20Intel%20GPU%20Perf/builds/740
,
May 9 2017
|
|||||||||||||
►
Sign in to add a comment |
|||||||||||||
Comment 1 by ashleymarie@chromium.org
, Mar 23 2017