New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 712739 link

Starred by 1 user

Issue metadata

Status: Fixed
Owner:
Closed: May 2017
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: Windows
Pri: 1
Type: Bug-Regression



Sign in to add a comment

V8_Fatal in Verifier::Run during WebglConformance_deqp_functional_gles3_texturefiltering_cube_combinations_* on Win NVIDIA

Project Member Reported by ynovikov@chromium.org, Apr 18 2017

Issue description

Started in https://build.chromium.org/p/chromium.gpu.fyi/builders/Win10%20Release%20%28NVIDIA%20Quadro%20P400%29/builds/521

Likely caused by V8 roll https://codereview.chromium.org/2818233004

Log:
#
# Fatal error in e:\b\c\b\win\src\v8\src\compiler\verifier.cc, line 1463
# Node #2875:Call has duplicate projections #2879 and #4101
#
Backtrace:
	base::debug::StackTrace::StackTrace [0x6C61C267+55]
	base::debug::StackTrace::StackTrace [0x6C61E3EA+10]
	V8_Fatal [0x6D268B61+81]
	v8::internal::compiler::Verifier::Run [0x6BFFBEAA+634]
	v8::internal::compiler::PipelineImpl::OptimizeGraph [0x6BFACBA4+1204]
	v8::internal::compiler::PipelineCompilationJob::ExecuteJobImpl [0x6BFAB581+17]
	v8::internal::OptimizingCompileDispatcher::CompileTask::Run [0x6BEC65D5+213]
	base::WorkerPool::RunsTasksOnCurrentThread [0x6C629A36+438]
	RtlSetThreadPreferredUILanguages [0x771C2614+1460]
	EtwNotificationRegister [0x7718CD41+1857]
	BaseThreadInitThunk [0x756438F4+36]
	RtlUnicodeStringToInteger [0x771C5DE3+595]
	RtlUnicodeStringToInteger [0x771C5DAE+542]
Backtrace:
	(No symbol) [0x00000000]
	v8::base::OS::Abort [0x6D26904D+13]
	v8::internal::compiler::Verifier::Run [0x6BFFBEAA+634]
	v8::internal::compiler::PipelineImpl::OptimizeGraph [0x6BFACBA4+1204]
	v8::internal::compiler::PipelineCompilationJob::ExecuteJobImpl [0x6BFAB581+17]
	v8::internal::CompilationJob::ExecuteJob [0x6BEC91F8+360]
	v8::internal::OptimizingCompileDispatcher::CompileTask::Run [0x6BEC65D5+213]
	base::WorkerPool::RunsTasksOnCurrentThread [0x6C629A36+438]
	RtlSetThreadPreferredUILanguages [0x771C2614+1460]
	EtwNotificationRegister [0x7718CD41+1857]
	BaseThreadInitThunk [0x756438F4+36]
	RtlUnicodeStringToInteger [0x771C5DE3+595]
	RtlUnicodeStringToInteger [0x771C5DAE+542]

The roll has a lot of CLs. machenbach@, could you please triage?
 
Cc: enne@chromium.org
Labels: Hotlist-PixelWrangler
Cc: mstarzinger@chromium.org jarin@chromium.org bmeu...@chromium.org
Components: Blink>JavaScript>Compiler
Owner: ----
Compiler folks, please triage.
Cc: -jarin@chromium.org jkummerow@chromium.org hablich@chromium.org
Owner: jarin@chromium.org
Status: Assigned (was: Untriaged)
Doesn't repro on Linux x64 with:

content/test/gpu/run_gpu_integration_test.py --test-filter 'WebglConformance_deqp_functional_gles3_texturefiltering_cube_combinations_' --verbose webgl_conformance --webgl-conformance-version=2.0.1 --extra-browser-args='--js-flags=--turbo-verify'

Maybe problem only shows on Windows?
Yes, the problem shows only on Win10 Release (NVIDIA Quadro P400).

Let's revert V8 to 6.0.2 and roll CLs one by one to see which one caused this?
Summary: V8_Fatal in Verifier::Run during WebglConformance_deqp_functional_gles3_texturefiltering_cube_combinations_* on Win NVIDIA (was: V8_Fatal in Verifier::Run during WebglConformance_deqp_functional_gles3_texturefiltering_cube_combinations_* on Win10 Release (NVIDIA Quadro P400) GPU.FYI)
Update:
This also flakes on Win7 Release (NVIDIA):
https://build.chromium.org/p/chromium.gpu.fyi/builders/Win7%20Release%20%28NVIDIA%29/builds/30020
https://build.chromium.org/p/chromium.gpu.fyi/builders/Win7%20Release%20%28NVIDIA%29/builds/29987
https://build.chromium.org/p/chromium.gpu.fyi/builders/Win7%20Release%20%28NVIDIA%29/builds/29955

This is severe, since it affects win_optional_gpu_tests_rel CQ bot, which also flaked here:
https://build.chromium.org/p/tryserver.chromium.win/builders/win_optional_gpu_tests_rel/builds/9311

Going to disable the tests until this is fixed.

Comment 6 by jarin@chromium.org, Apr 21 2017

I am investigating, but I will likely be able to have a serious look only on Monday, when I get access to Win10 machine.
Tests marked as Flaky in https://codereview.chromium.org/2831123002/, not sure why commit-bot didn't update this CL.

Comment 8 by jarin@chromium.org, Apr 25 2017

Cc: cbruni@chromium.org

Comment 9 by jarin@chromium.org, Apr 26 2017

We tried to reproduce on a Win10 box with x86 build, but the test does not crash for us (we tried to run the full test suite several times).

I believe we would have high chance of finding the cause if we managed to get a run with a log from --js-flags="--turbo-verify --trace-turbo-graph". Would it be possible to get such a log either from the bots or from someone who can repro locally?

Comment 10 by kbr@chromium.org, Apr 26 2017

Yes, we can add these to the bots. We'll do this in src/content/test/gpu/generate_buildbot_json.py and then run that script to generate the associated JSON files. I'll put up a CL.

Let me ask though: how large do you expect the logs to be with these flags turned on?

These are huge. Just for --test-filter 'WebglConformance_deqp_functional_gles3_texturefiltering_cube_combinations_' stdout is 716MB.

Maybe it's better to try to repro directly on Win10 Release (NVIDIA Quadro P400) bot.

Comment 12 by kbr@chromium.org, Apr 26 2017

Thanks Yuly for confirming that.

Jarin, I'm sorry but that's a non-starter for turning it on even on this bot. It'll surely break stuff.

Instead, you can log on to this bot using the "Chrome Internal GPU Pixel Wrangling Instructions" linked from this page:
https://www.chromium.org/developers/how-tos/gpu-wrangling

Feel free to take it offline any time for your own testing. Note that you can upload your own isolates to the isolate server and run them on that machine; that's probably the easiest way to transfer binaries to it.

Alternatively, if you want to add more flags to V8 to reduce the size of the logs (maybe only dumping things if TurboFan verification actually failed?), we could consider doing that.

Project Member

Comment 13 by bugdroid1@chromium.org, May 4 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/v8/v8.git/+/102606e8213b43af1b21d1142b1fe790a0212c31

commit 102606e8213b43af1b21d1142b1fe790a0212c31
Author: jarin <jarin@chromium.org>
Date: Thu May 04 10:47:45 2017

[turbofan] Fix verification of duplicate projections.

BUG= chromium:712739 

Review-Url: https://codereview.chromium.org/2857983004
Cr-Commit-Position: refs/heads/master@{#45081}

[modify] https://crrev.com/102606e8213b43af1b21d1142b1fe790a0212c31/src/compiler/verifier.cc

Status: Fixed (was: Assigned)
Project Member

Comment 15 by bugdroid1@chromium.org, May 8 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/879fa3502244002049f6bc9106e4df14d08f1515

commit 879fa3502244002049f6bc9106e4df14d08f1515
Author: ynovikov <ynovikov@chromium.org>
Date: Mon May 08 20:59:24 2017

Update WebGL2 expectations

deqp/functional/gles3/texturefiltering/cube_combinations_*
should pass since
https://codereview.chromium.org/2863613002/

BUG= 712739 
TBR=kbr@chromium.org
CQ_INCLUDE_TRYBOTS=master.tryserver.chromium.android:android_optional_gpu_tests_rel;master.tryserver.chromium.linux:linux_optional_gpu_tests_rel;master.tryserver.chromium.mac:mac_optional_gpu_tests_rel;master.tryserver.chromium.win:win_optional_gpu_tests_rel

Review-Url: https://codereview.chromium.org/2867013005
Cr-Commit-Position: refs/heads/master@{#470120}

[modify] https://crrev.com/879fa3502244002049f6bc9106e4df14d08f1515/content/test/gpu/gpu_tests/webgl2_conformance_expectations.py

Sign in to add a comment