New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 735527 link

Starred by 3 users

Issue metadata

Status: Duplicate
Merged: issue 725664
Owner:
Closed: Jun 2017
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: Windows
Pri: 1
Type: Bug-Regression



Sign in to add a comment

WebGL2 3D texture filtering tests crashing with OOM inside V8 compiler on some Windows bots

Project Member Reported by geoffl...@chromium.org, Jun 21 2017

Issue description

deqp/functional/gles3/texturefiltering/3d_sizes_03.html and deqp/functional/gles3/texturefiltering/3d_formats_05.html and consistently failing on the Win 7 Release (AMD) bot since build #983 (https://build.chromium.org/p/chromium.gpu.fyi/builders/Win7%20Release%20%28AMD%20R7%20240%29/builds/983).

The test appears to crash in v8:
351eefd8 6731a50c e0000008 00000001 00000001 KERNELBASE!RaiseException+0x58
	351eeff0 67343da5 00100000 351ef014 672ceadf chrome_child!base::EnableTerminationOnOutOfMemory+0x3c
	351eeffc 672ceadf 00100000 00100000 00000008 chrome_child!base::allocator::WinCallNewHandler+0x15
	351ef014 670e9a1b 00100000 00000008 019b27b0 chrome_child!malloc+0x2f
	351ef02c 670e9ed4 00100000 019b27b0 00000008 chrome_child!v8::internal::AccountingAllocator::GetSegment+0x1b
	351ef044 670e9e18 00100000 00000008 019b27b0 chrome_child!v8::internal::Zone::NewSegment+0x14
	351ef058 670e9d24 00000008 ffbb0010 0d69fab0 chrome_child!v8::internal::Zone::NewExpand+0x98
	351ef06c 66c8cc17 00000008 351ef1fc 0d69fb00 chrome_child!v8::internal::Zone::New+0x34
	351ef084 66c8d0a9 019b27b0 0d69fab0 0d69fa01 chrome_child!v8::internal::compiler::BranchElimination::ControlPathConditions::AddCondition+0x47
	351ef0c4 66c8cd1d 351ef0f8 0d69fa74 07886c64 chrome_child!v8::internal::compiler::BranchElimination::ReduceDeoptimizeConditional+0xb9
	351ef0dc 66cc5e35 351ef0f8 0d69fb00 351ef264 chrome_child!v8::internal::compiler::BranchElimination::Reduce+0x3d
	351ef0fc 66cc61a2 351ef1c0 0d69fb00 351ef264 chrome_child!v8::internal::compiler::GraphReducer::Reduce+0x25
	351ef1cc 66cc5ed5 019b27b0 019b27b0 054b460c chrome_child!v8::internal::compiler::GraphReducer::ReduceTop+0x1b2
	351ef1e4 66cc5e9b 078bd968 66d44859 00000000 chrome_child!v8::internal::compiler::GraphReducer::ReduceNode+0x35
	351ef1ec 66d44859 00000000 054b46bc 695d8e44 chrome_child!v8::internal::compiler::GraphReducer::ReduceGraph+0xb
	351ef2e4 66d43437 054b460c 019b27b0 055b4df8 chrome_child!v8::internal::compiler::LateOptimizationPhase::Run+0x289
	351ef30c 66d41ac1 076b8588 66c77446 078ea628 chrome_child!v8::internal::compiler::PipelineImpl::OptimizeGraph+0x777
	351ef314 66c77446 078ea628 6a105ea0 5c59381b chrome_child!v8::internal::compiler::PipelineCompilationJob::ExecuteJobImpl+0x11
	351ef35c 66c746a2 05ec9090 696d533c 69adc0d4 chrome_child!v8::internal::CompilationJob::ExecuteJob+0x126
	351ef3b0 673573f9 054938c8 00000000 68142c9d chrome_child!v8::internal::OptimizingCompileDispatcher::CompileTask::Run+0xd2
	351ef498 6734a454 696d5364 05ec9090 05ec68e8 chrome_child!base::debug::TaskAnnotator::RunTask+0x199
	351ef6c0 6734a835 05ec9090 05ec68e8 00000000 chrome_child!base::internal::TaskTracker::PerformRunTask+0x434
	351ef790 6736b059 00ec68e8 05d3ff20 77031430 chrome_child!base::internal::TaskTracker::RunNextTask+0xe5
	351ef874 672ef052 00000000 00000000 05d3ff20 chrome_child!base::internal::SchedulerWorker::Thread::ThreadMain+0x409
	*** WARNING: Unable to verify checksum for kernel32.dll
	*** ERROR: Symbol file could not be found.  Defaulted to export symbols for kernel32.dll - 
	351ef890 7703338a 0000039c 351ef8dc 77a69902 chrome_child!base::PlatformThread::Sleep+0x132
	351ef89c 77a69902 05d3ff20 4237d5fd 00000000 kernel32!BaseThreadInitThunk+0x12
	351ef8dc 77a698d5 672eefc0 05d3ff20 ffffffff ntdll!RtlInitializeExceptionChain+0x63
	351ef8f4 00000000 672eefc0 05d3ff20 00000000 ntdll!RtlInitializeExceptionChain+0x36
	
I can't reproduce locally with Win10 AMD (will try Win 7 next).

There is one v8 roll in the regression range (https://codereview.chromium.org/2950943003) but it's CLs don't look too suspect.

There is also one ANGLE CL in the regression range (https://chromium-review.googlesource.com/539536) but it also does not look suspect.

 

Comment 1 by kbr@chromium.org, Jun 21 2017

Cc: machenb...@chromium.org bmeu...@chromium.org
Could someone from the V8 team take a look at this?

I think these 3D texture tests may be allocating a ton of memory and pushing things close to the edge.

Cc: mvstan...@chromium.org u...@chromium.org
+ v8 mem sheriffs. Though there's a lot of compiler code on the stack... bmeurer might know more.

Comment 3 by kbr@chromium.org, Jun 23 2017

Labels: -Type-Bug Type-Bug-Regression
Owner: bmeu...@chromium.org
Status: Assigned (was: Untriaged)
V8 team: this is new behavior on these two bots:

https://luci-milo.appspot.com/buildbot/chromium.gpu.fyi/Win10%20Release%20%28NVIDIA%20Quadro%20P400%29/?limit=200
https://luci-milo.appspot.com/buildbot/chromium.gpu.fyi/Win7%20Release%20%28AMD%20R7%20240%29/?limit=200

and since the crashing tests are pretty random, it's difficult to add flaky suppressions.

bmeurer@, since this OOM's happening in the V8 compiler could you please investigate this and see whether the maximum amount of memory used during compilation has increased?

Comment 4 by kbr@chromium.org, Jun 23 2017

Summary: WebGL2 3D texture filtering tests crashing with OOM inside V8 compiler on some Windows bots (was: WebGL2 3D texture filtering tests failing on Win 7 Release (AMD))

Comment 5 by kbr@chromium.org, Jun 23 2017

Note that we could probably start by marking just these tests flaky:

WebglConformance_deqp_functional_gles3_texturefiltering_3d_formats_05
WebglConformance_deqp_functional_gles3_texturefiltering_3d_formats_06
WebglConformance_deqp_functional_gles3_texturefiltering_3d_sizes_02
WebglConformance_deqp_functional_gles3_texturefiltering_3d_sizes_03

Comment 6 by kbr@chromium.org, Jun 23 2017

One more, from the AMD bot:

WebglConformance_deqp_functional_gles3_texturefiltering_3d_formats_07

Comment 8 by kbr@chromium.org, Jun 26 2017

Cc: hablich@chromium.org
Suppressing these flaky failures in https://chromium-review.googlesource.com/548100 .

Could the V8 team please try to reproduce these on Windows in house?

Comment 9 by kbr@chromium.org, Jun 26 2017

Cc: geoffl...@chromium.org
 Issue 736537  has been merged into this issue.
Project Member

Comment 10 by bugdroid1@chromium.org, Jun 26 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/83354dada1b661bde9c996e336fbc23ddeb74ea9

commit 83354dada1b661bde9c996e336fbc23ddeb74ea9
Author: Kenneth Russell <kbr@chromium.org>
Date: Mon Jun 26 18:29:41 2017

Suppress flaky OOMs in texturefiltering tests on Windows.

BUG= 735527 
TBR=zmo@chromium.org

Cq-Include-Trybots: master.tryserver.chromium.android:android_optional_gpu_tests_rel;master.tryserver.chromium.linux:linux_optional_gpu_tests_rel;master.tryserver.chromium.mac:mac_optional_gpu_tests_rel;master.tryserver.chromium.win:win_optional_gpu_tests_rel
Change-Id: I55988743fc55439ba87c70936c4ae352a9aa067d
Reviewed-on: https://chromium-review.googlesource.com/548100
Commit-Queue: Kenneth Russell <kbr@chromium.org>
Reviewed-by: Kenneth Russell <kbr@chromium.org>
Cr-Commit-Position: refs/heads/master@{#482330}
[modify] https://crrev.com/83354dada1b661bde9c996e336fbc23ddeb74ea9/content/test/gpu/gpu_tests/webgl2_conformance_expectations.py

Mergedinto: 725664
Status: Duplicate (was: Assigned)

Sign in to add a comment