New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 831145 link

Starred by 4 users

Issue metadata

Status: Fixed
Owner:
Closed: Apr 2018
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: Android
Pri: 1
Type: Bug

Blocking:
issue 828582



Sign in to add a comment

telemetry_perf_unittests benchmarks.system_health_smoke_test.SystemHealthBenchmarkSmokeTest.system_health.memory_mobile.browse:shopping:avito times out with recent clang

Project Member Reported by h...@chromium.org, Apr 10 2018

Issue description

Using e.g. the Clang version packaged here:
https://chromium-review.googlesource.com/c/chromium/src/+/1000784

The test fails consistently on the android_n5x_swarming_rel trybot:
https://ci.chromium.org/buildbot/tryserver.chromium.android/android_n5x_swarming_rel/397363

It seems like it's timing out. This is blocking the Clang roll.

I haven't been able to reproduce the failure by building locally and running on swarming. In my most successful attempt at running anything, it passes: https://chromium-swarm.appspot.com/task?id=3cc8438589538110&refresh=10&show_raw=1

Lots of similar tests are disabled due to timing out (see https://cs.chromium.org/chromium/src/tools/perf/benchmarks/system_health_smoke_test.py?l=38). I'm starting to think we should disable this one, as the failure is completely inactionable.
 

Comment 1 by h...@chromium.org, Apr 10 2018

These are my notes from trying to run the test, mostly based on cargo-culting command lines from the buildbot:

args.gn:

dcheck_always_on = true
ffmpeg_branding = "Chrome"
is_component_build = false
is_debug = false
proprietary_codecs = true
strip_absolute_paths_from_debug_symbols = true
symbol_level = 0
target_cpu = "arm64"
target_os = "android"
use_goma = true

$ ninja -C out/release -j800 chrome_public_apk
$ tools/mb/mb.py isolate //out/release telemetry_perf_unittests
$ tools/swarming_client/isolate.py archive -I https://isolateserver.appspot.com -i out/release/telemetry_perf_unittests.isolate -s out/release/telemetry_perf_unittests.isolated --verbose
(got hash: bfa20de76d8f9fa9a0945d0ff926c7cb579411c9)
$ tools/swarming_client/swarming.py trigger --swarming https://chromium-swarm.appspot.com --isolate-server https://isolateserver.appspot.com  --dimension device_os MMB29Q --dimension device_type bullhead --dimension os Android --dimension pool Chrome --named-cache swarming_module_cache_vpython .swarming_module_cache/vpython  --cipd-package '.swarming_module:infra/python/cpython/${platform}:version:2.7.14.chromium14' --cipd-package '.swarming_module:infra/tools/luci/logdog/butler/${platform}:git_revision:e1abc57be62d198b5c2f487bfb2fa2d2eb0e867c' --cipd-package '.swarming_module:infra/tools/luci/vpython-native/${platform}:git_revision:e1abc57be62d198b5c2f487bfb2fa2d2eb0e867c' --cipd-package '.swarming_module:infra/tools/luci/vpython/${platform}:git_revision:e1abc57be62d198b5c2f487bfb2fa2d2eb0e867c' --env-prefix PATH .swarming_module --env-prefix PATH .swarming_module/bin --env-prefix VPYTHON_VIRTUALENV_ROOT .swarming_module_cache/vpython --isolated bfa20de76d8f9fa9a0945d0ff926c7cb579411c9 -- '--device=android' '--isolated-script-test-output=${ISOLATED_OUTDIR}/output.json' '--isolated-script-test-perf-output=${ISOLATED_OUTDIR}/perftest-output.json' --isolated-script-test-filter=benchmarks.system_health_smoke_test.SystemHealthBenchmarkSmokeTest.system_health.memory_mobile.browse:shopping:avito

This is the job: https://chromium-swarm.appspot.com/task?id=3cc8438589538110

Comment 2 by thakis@chromium.org, Apr 10 2018

Cc: dpranke@chromium.org
Dirk, do we have docs on how to trigger swarming runs of telemetry_perf_unittests locally? We know how to do that for non-script tests, and we tried to get it going (see comment 1), but the tests fairly consistently fail on the cq for our clang roll, but don't repro when trying to trigger them from a local build dir, despite using the same compiler and args.gn.

Comment 3 by h...@chromium.org, Apr 10 2018

Here are two other telemetry_perf_unittests failures from looking at recent builds:

benchmarks.system_health_smoke_test.SystemHealthBenchmarkSmokeTest.system_health.memory_mobile.background:media:imgur
https://ci.chromium.org/buildbot/tryserver.chromium.android/android_n5x_swarming_rel/397335

benchmarks.system_health_smoke_test.SystemHealthBenchmarkSmokeTest.system_health.memory_mobile.browse:news:qq
https://ci.chromium.org/buildbot/tryserver.chromium.android/android_n5x_swarming_rel/396569

This feel like a very fragile test target in general, especially iiuc that this is supposed to unit-test the performance testing framework, not do perf tests in itself.

It's possible clang changed something, pushing memory_mobile.browse:shopping:avito over the timeout-edge when run on the bot, but even so I think the fix is to disable the test.
Cc: eyaich@chromium.org
I don't know offhand what might be wrong with that. 

nednguyen@, eyaich@, any ideas?
Cc: perezju@chromium.org
Looks like there is a crash, then telemetry fails at symbolize the crash:

[9/17] benchmarks.system_health_smoke_test.SystemHealthBenchmarkSmokeTest.system_health.memory_mobile.browse:shopping:avito queuedTraceback (most recent call last):
  File "/b/swarming/w/ir/third_party/android_platform/development/scripts/stack.py", line 247, in <module>
    sys.exit(main(sys.argv[1:]))
  File "/b/swarming/w/ir/third_party/android_platform/development/scripts/stack.py", line 238, in main
    arch_defined, symbolizer)
  File "/b/swarming/w/ir/third_party/android_platform/development/scripts/stack_core.py", line 192, in ConvertTrace
    ResolveCrashSymbol(list(useful_log), more_info, llvm_symbolizer)
  File "/b/swarming/w/ir/third_party/android_platform/development/scripts/stack_core.py", line 391, in ResolveCrashSymbol
    os.path.join(symbol.SYMBOLS_DIR, symbol.TranslateLibPath(area)),
  File "/b/swarming/w/ir/third_party/android_platform/development/scripts/symbol.py", line 335, in TranslateLibPath
    mapping = MapDeviceApkToLibrary(apk)
  File "/b/swarming/w/ir/third_party/android_platform/development/scripts/symbol.py", line 282, in MapDeviceApkToLibrary
    matching_apks = GetMatchingApks(device_apk_name)
  File "/b/swarming/w/ir/third_party/android_platform/development/scripts/symbol.py", line 270, in GetMatchingApks
    _GetApkPackageName(apk_path) == package_name)]
  File "/b/swarming/w/ir/third_party/android_platform/development/scripts/symbol.py", line 160, in _GetApkPackageName
    aapt_path = _GetAaptPath()
  File "/b/swarming/w/ir/third_party/android_platform/development/scripts/symbol.py", line 139, in _GetAaptPath
    raise Exception('Missing binary: ' + aapt_exe)
Exception: Missing binary: /b/swarming/w/ir/third_party/android_tools/sdk/build-tools/27.0.3/aapt
Traceback (most recent call last):
  File "/b/swarming/w/ir/third_party/android_platform/development/scripts/stack.py", line 247, in <module>
    sys.exit(main(sys.argv[1:]))
  File "/b/swarming/w/ir/third_party/android_platform/development/scripts/stack.py", line 238, in main
    arch_defined, symbolizer)
  File "/b/swarming/w/ir/third_party/android_platform/development/scripts/stack_core.py", line 192, in ConvertTrace
    ResolveCrashSymbol(list(useful_log), more_info, llvm_symbolizer)
  File "/b/swarming/w/ir/third_party/android_platform/development/scripts/stack_core.py", line 391, in ResolveCrashSymbol
    os.path.join(symbol.SYMBOLS_DIR, symbol.TranslateLibPath(area)),
  File "/b/swarming/w/ir/third_party/android_platform/development/scripts/symbol.py", line 335, in TranslateLibPath
    mapping = MapDeviceApkToLibrary(apk)
  File "/b/swarming/w/ir/third_party/android_platform/development/scripts/symbol.py", line 282, in MapDeviceApkToLibrary
    matching_apks = GetMatchingApks(device_apk_name)
  File "/b/swarming/w/ir/third_party/android_platform/development/scripts/symbol.py", line 270, in GetMatchingApks
    _GetApkPackageName(apk_path) == package_name)]
  File "/b/swarming/w/ir/third_party/android_platform/development/scripts/symbol.py", line 160, in _GetApkPackageName
    aapt_path = _GetAaptPath()
  File "/b/swarming/w/ir/third_party/android_platform/development/scripts/symbol.py", line 139, in _GetAaptPath
    raise Exception('Missing binary: ' + aapt_exe)
Exception: Missing binary: /b/swarming/w/ir/third_party/android_tools/sdk/build-tools/27.0.3/aapt

Nico: here is the instruction for reproducing Telemetry failure: go/reproduce-telemetry

Comment 7 by h...@chromium.org, Apr 10 2018

> Looks like there is a crash, then telemetry fails at symbolize the crash:

From https://logs.chromium.org/v/?s=chromium%2Fbb%2Ftryserver.chromium.android%2Fandroid_n5x_swarming_rel%2F397363%2F%2B%2Frecipes%2Fsteps%2Ftelemetry_perf_unittests__with_patch__on_Android%2F0%2Fstdout

It's really hard to parse the output, but I guess it comes down to this:

 File "/b/swarming/w/ir/third_party/catapult/telemetry/third_party/websocket-client/websocket/_socket.py", line 83, in recv
      raise WebSocketTimeoutException(message)
  TimeoutException: 
  ********************************************************************************
  (/b/swarming/w/ir/third_party/catapult/telemetry/telemetry/internal/backends/chrome_inspector/inspector_backend.py:487 _ConvertExceptionFromInspectorWebsocket) The app is probably crashed:
  

Oh hey, further down there's something useful:

  	--------- beginning of crash
  	04-10 08:52:42.437 31552 31565 F libc    : Fatal signal 6 (SIGABRT), code -6 in tid 31565 (CrRendererMain)
  	04-10 08:52:42.556   494   494 I SELinux : SELinux: Loaded file_contexts contexts from /file_contexts.
  	04-10 08:52:42.560   494   494 F DEBUG   : *** *** *** *** *** *** *** *** *** *** *** *** *** *** *** ***
  	04-10 08:52:42.560   494   494 F DEBUG   : Build fingerprint: 'google/bullhead/bullhead:6.0.1/MMB29Q/2480792:userdebug/dev-keys'
  	04-10 08:52:42.560   494   494 F DEBUG   : Revision: 'rev_1.0'
  	04-10 08:52:42.560   494   494 F DEBUG   : ABI: 'arm64'
  	04-10 08:52:42.560   494   494 F DEBUG   : pid: 31552, tid: 31565, name: CrRendererMain  >>> org.chromium.chrome:sandboxed_process0 <<<
  	04-10 08:52:42.560   494   494 F DEBUG   : signal 6 (SIGABRT), code -6 (SI_TKILL), fault addr --------
  	04-10 08:52:42.579   494   494 F DEBUG   : Abort message: '[FATAL:ax_position.cc(90)] Check failed: container.GetNode() && container.GetNode()->IsTextNode(). Text positions should be anchored to a text node.
  	04-10 08:52:42.579   494   494 F DEBUG   : #00 0x0000007f8be19367 /data/app/org.chromium.chrome-1/base.apk+0x000000000297a367
  	04-10 08:52:42.579   494   494 F DEBUG   : #01 0x0000007f9004a7a3 /data/app/org.chromium.chrome-1/base.apk+0x0000000006bab7a3
  	04-10 08:52:42.579   494   494 F DEBUG   : #02 0x0000007f900279bf /data/app/org.chromium.chrome-1/base.apk+0x0000000006b889bf
  	04-10 08:52:42.579   494   494 F DEBUG   : #03 0x0000007f900ce1d7 /data/app/org.chromium.chrome-1/base.apk+0x0000000006c2f1d7
  	04-10 08:52:42.579   494   494 F DEBUG   : #04 0x0000007f
  	04-10 08:52:42.580   494   494 F DEBUG   :     x0   0000000000000000  x1   0000000000007b4d  x2   0000000000000006  x3   0000000000000000
  	04-10 08:52:42.580   494   494 F DEBUG   :     x4   0000000000000000  x5   0000000000000001  x6   0000000000000000  x7   0000000000000000
  	04-10 08:52:42.580   494   494 F DEBUG   :     x8   0000000000000083  x9   31c38929b3c14721  x10  0000000000000056  x11  0000007fa5ddeed3
  	04-10 08:52:42.580   494   494 F DEBUG   :     x12  0000007fa5ddeed3  x13  0000007fa5ddf19a  x14  0000007f8a9ed8c7  x15  0000000000000054
  	04-10 08:52:42.581   494   494 F DEBUG   :     x16  0000007faf0d7568  x17  0000007faf06a13c  x18  0000007fa5ddeede  x19  0000007fa5de5500
  	04-10 08:52:42.581   494   494 F DEBUG   :     x20  0000007fa5de5440  x21  0000000000000000  x22  0000000000000006  x23  0000007fa5de54f0
  	04-10 08:52:42.581   494   494 F DEBUG   :     x24  0000007fa5ddf2d9  x25  0000007fa5de54f0  x26  0000007fa5ddf2d8  x27  0000007fa5de54f0
  	04-10 08:52:42.581   494   494 F DEBUG   :     x28  00000000000000dc  x29  0000007fa5ddf200  x30  0000007faf0678d8
  	04-10 08:52:42.581   494   494 F DEBUG   :     sp   0000007fa5ddf200  pc   0000007faf06a144  pstate 0000000020000000
  	04-10 08:52:42.593   494   494 F DEBUG   : 
  	04-10 08:52:42.593   494   494 F DEBUG   : backtrace:
  	04-10 08:52:42.593   494   494 F DEBUG   :     #00 pc 000000000006a144  /system/lib64/libc.so (tgkill+8)
  	04-10 08:52:42.593   494   494 F DEBUG   :     #01 pc 00000000000678d4  /system/lib64/libc.so (pthread_kill+68)
  	04-10 08:52:42.594   494   494 F DEBUG   :     #02 pc 0000000000023838  /system/lib64/libc.so (raise+28)
  	04-10 08:52:42.594   494   494 F DEBUG   :     #03 pc 000000000001dfd8  /system/lib64/libc.so (abort+60)
  	04-10 08:52:42.594   494   494 F DEBUG   :     #04 pc 0000000000026c14  /data/app/org.chromium.chrome-1/base.apk (offset 0x2937000)
  	********************************************************************************


So it's hitting a DCHECK? Why does the output have to be so noisy :-(



> Nico: here is the instruction for reproducing Telemetry failure: go/reproduce-telemetry

We were looking for instructions for reproducing Telemetry failures on swarming, not running locally. I.e. build locally, run on swarming like the bot. Do we have instructions for that?

Comment 8 by h...@chromium.org, Apr 10 2018

Dirk or Ned: Can you help provide the steps needed to repro this with a local build, i.e.

1. Patch in https://chromium-review.googlesource.com/c/chromium/src/+/1000784
2. Sync
3. Build
4. Run the failing test on swarming.

With that I can bisect LLVM, but so far I've failed at reproducing.
hans@ if you ve been able to run the failing test locally but can't reproduce the failure, I am guessing that it's because your local hardware is different from the swarming bot? 

Do you have a nexus5x device to reproduce?
nednguyen: We're trying to build locally and then run the test on swarming for bisecting, see comment 8. We are not interested in testing locally. We are unable to repro the failure on swarming, even though it consistently happens on swarming when the tests are run via cq. We're trying to replicate that failure, without the cq.
#10: How are you build locally and  run the test on swarming? I didn't know that you could do this :-)

Maybe starting by replicating the failure first. Can you make sure that the binary you built locally is exactly the same as the one in the isolate?
See steps in comment 1 on how we do this. It's supposed to be the same what the bots do, which also build somewhere and then upload the binary to swarming. This works fine for us for non-script test binaries.

The isolate we use (when we try to repro) is by packing up the locally-built binary, so that will be identical. It won't be identical to the binary used by the cq I'd guess, since our build probably isn't 100% deterministic.
I see. I am not an expert of the toolchain, so I defer to Dirk on this. I wonder whether the failure to reproduce on swarming is due to build determinism (which sounds like it would be a tough bug to diagnose) :-/
The test consistently fails when run on the cq. Even if the binary is different each time in boring ways (not codegen), the test consistently fails. This is not due to build determinism.

Who knows how the telemetry perf unittests are run?
I know how telemetry perf unittests are run. What particular questions are you having?
The question is: How do I trigger a swarming run of telemetry_perf_unittests from my local machine?

(Kind of the reverse of what go/reproduce-telemetry describes)
On https://ci.chromium.org/buildbot/tryserver.chromium.android/android_n5x_swarming_rel/397363 , "test_pre_run.[trigger] telemetry_perf_unittests (with patch)"'s stdout has a swarming trigger invocation that I copied and slightly tweaked, and which I'm now running to try and reproduce the original failure at first:

$ python tools/swarming_client/swarming.py trigger --swarming https://chromium-swarm.appspot.com --isolate-server https://isolateserver.appspot.com --priority 30 --shards 15 --task-name 'telemetry_perf_unittests (with patch rerun)/Android/f34c255617/android_n5x_swarming_rel/397363' --dump-json /tmp/tmpwP8ajA.json --expiration 3600 --io-timeout 1200 --hard-timeout 1500 --dimension device_os MMB29Q --dimension device_type bullhead --dimension os Android --dimension pool Chrome --dimension temp_band '<30' --named-cache swarming_module_cache_vpython .swarming_module_cache/vpython --tag build_is_experimental:false --tag buildername:android_n5x_swarming_rel --tag buildnumber:397363 --tag data:f34c255617c3d6a74802cc6a3b44e686a65b2eb7 --tag gerrit:https://chromium-review.googlesource.com/c/1000784/1 --tag master:tryserver.chromium.android --tag name:telemetry_perf_unittests --tag patch_project:chromium/src --tag project:chromium --tag purpose:CQ --tag purpose:pre-commit --tag slavename:slave657-c4 --tag 'stepname:telemetry_perf_unittests (with patch rerun) on Android' --user thakis@chromium.org --cipd-package '.swarming_module:infra/python/cpython/${platform}:version:2.7.14.chromium14' --cipd-package '.swarming_module:infra/tools/luci/logdog/butler/${platform}:git_revision:e1abc57be62d198b5c2f487bfb2fa2d2eb0e867c' --cipd-package '.swarming_module:infra/tools/luci/vpython-native/${platform}:git_revision:e1abc57be62d198b5c2f487bfb2fa2d2eb0e867c' --cipd-package '.swarming_module:infra/tools/luci/vpython/${platform}:git_revision:e1abc57be62d198b5c2f487bfb2fa2d2eb0e867c' --env-prefix PATH .swarming_module --env-prefix PATH .swarming_module/bin --env-prefix VPYTHON_VIRTUALENV_ROOT .swarming_module_cache/vpython --isolated f34c255617c3d6a74802cc6a3b44e686a65b2eb7 -- '--browser=android-chromium' '--device=android' '--isolated-script-test-output=${ISOLATED_OUTDIR}/output.json' '--isolated-script-test-perf-output=${ISOLATED_OUTDIR}/perftest-output.json'

Triggered task: telemetry_perf_unittests (with patch rerun)/Android/f34c255617/android_n5x_swarming_rel/397363
To collect results, use:
  swarming.py collect -S https://chromium-swarm.appspot.com --json /tmp/tmpwP8ajA.json
Or visit:
  https://chromium-swarm.appspot.com/user/task/3ccad848896d2910
  https://chromium-swarm.appspot.com/user/task/3ccad849ab2c6310
  https://chromium-swarm.appspot.com/user/task/3ccad84aa4640210
  https://chromium-swarm.appspot.com/user/task/3ccad84ba7867210
  https://chromium-swarm.appspot.com/user/task/3ccad84d530bba10
  https://chromium-swarm.appspot.com/user/task/3ccad84e8b26bc10
  https://chromium-swarm.appspot.com/user/task/3ccad84fbf33bb10
  https://chromium-swarm.appspot.com/user/task/3ccad850e08afe10
  https://chromium-swarm.appspot.com/user/task/3ccad8522e41d310
  https://chromium-swarm.appspot.com/user/task/3ccad85352d2c210
  https://chromium-swarm.appspot.com/user/task/3ccad854fb11cf10
  https://chromium-swarm.appspot.com/user/task/3ccad855da890b10
  https://chromium-swarm.appspot.com/user/task/3ccad85773b9d310
  https://chromium-swarm.appspot.com/user/task/3ccad858b9a1bb10
  https://chromium-swarm.appspot.com/user/task/3ccad859b2f66d10
With linebreaks between args for readability:

--swarming https://chromium-swarm.appspot.com
--isolate-server https://isolateserver.appspot.com
--priority 30
--shards 15
--task-name 'foo /Anid/f34c255617/android_n5x_swarming_rel/397363'
--dump-json /tmp/tmpwP8ajA.json
--expiration 3600
--io-timeout 1200
--hard-timeout 1500
--dimension device_os MMB29Q
--dimension device_type bullhead
--dimension os Android
--dimension pool Chrome
--dimension temp_band '<30'
--named-cache swarming_module_cache_vpython .swarming_module_cache/vpython
--tag build_is_experimental:false
--tag buildername:android_n5x_swarming_rel
--tag buildnumber:397363
--tag data:f34c255617c3d6a74802cc6a3b44e686a65b2eb7
--tag gerrit:https://chromium-review.googlesource.com/c/1000784/1
--tag master:tryserver.chromium.android
--tag name:telemetry_perf_unittests
--tag patch_project:chromium/src
--tag project:chromium
--tag purpose:CQ
--tag purpose:pre-commit
--tag slavename:slave657-c4
--tag 'stepname:telemetry_perf_unittests (with patch rerun) on Android'
--user thakis@chromium.org
--cipd-package '.swarming_module:infra/python/cpython/${platform}:version:2.7.14.chromium14'
--cipd-package '.swarming_module:infra/tools/luci/logdog/butler/${platform}:git_revision:e1abc57be62d198b5c2f487bfb2fa2d2eb0e867c'
--cipd-package '.swarming_module:infra/tools/luci/vpython-native/${platform}:git_revision:e1abc57be62d198b5c2f487bfb2fa2d2eb0e867c'
--cipd-package '.swarming_module:infra/tools/luci/vpython/${platform}:git_revision:e1abc57be62d198b5c2f487bfb2fa2d2eb0e867c'
--env-prefix PATH .swarming_module
--env-prefix PATH .swarming_module/bin
--env-prefix VPYTHON_VIRTUALENV_ROOT .swarming_module_cache/vpython
--isolated f34c255617c3d6a74802cc6a3b44e686a65b2eb7
--
'--browser=android-chromium'
'--device=android'
'--isolated-script-test-output=${ISOLATED_OUTDIR}/output.json'
'--isolated-script-test-perf-output=${ISOLATED_OUTDIR}/perftest-output.json'

(which looks very to hans's notes in comment 1; we'll see what the results are)
Cc: mar...@chromium.org
+maruel who may help with trigger tests on swarming bot
I then ran `tools/swarming_client/swarming.py collect -S https://chromium-swarm.appspot.com --json /tmp/tmpwP8ajA.json > output.txt` and that output.txt has the

        --------- beginning of crash

line mentioned in comment 7. So I managed to repro with the cq-built binary. Now to try with a locally-built one...
`testing/scripts/telemetry_perf_unittests.py compile_targets --output -` prints ["chrome"], so I'm building target chrome (from https://logs.chromium.org/v/?s=chromium%2Fbb%2Ftryserver.chromium.android%2Fandroid_n5x_swarming_rel%2F397363%2F%2B%2Frecipes%2Fsteps%2Fget_compile_targets_for_scripts%2F0%2Fstdout)
...that target doesn't exist in an android build; I'll build chrome_public_apk instead like Hans did. I wonder how that get_compile_targets_for_scripts step works.
Building and then triggering (the mb isolate command ended up building more stuff; I guess it knows what targets to build? but that also suggests that this is not exactly how the bots upload isolated files) with the same command as before just with a different --isolated flag:

thakis@thakis:~/src/chrome/src$ time ninja -C out/gnand chrome_public_apk -j1000
ninja: Entering directory `out/gnand'
[32286/32286] STAMP obj/chrome/android/chrome_public_apk.stamp

real    7m38.259s
user    77m59.872s
sys     5m32.904s
thakis@thakis:~/src/chrome/src$ tools/mb/mb.py isolate out/gnand telemetry_perf_unittests
ninja -C out/gnand telemetry_perf_unittests -j 960 -l 48
ninja: Entering directory `out/gnand'
[23613/23613] STAMP obj/chrome/test/telemetry_perf_unittests.stamp
python tools/swarming_client/isolate.py check -i out/gnand/telemetry_perf_unittests.isolate -s out/gnand/telemetry_perf_unittests.isolated
thakis@thakis:~/src/chrome/src$ tools/swarming_client/isolate.py archive -I https://isolateserver.appspot.com -i out/gnand/telemetry_perf_unittests.isolate -s out/gnand/telemetry_perf_unittests.isolated 
0b65cf924c2ceab2754ecf48e06a0633adc38b20  telemetry_perf_unittests

thakis@thakis:~/src/chrome/src$ 
thakis@thakis:~/src/chrome/src$ python tools/swarming_client/swarming.py trigger --swarming https://chromium-swarm.appspot.com --isolate-server https://isolateserver.appspot.com --priority 30 --shards 15 --task-name 'telemetry_perf_unittests (with patch rerun)/Android/f34c255617/android_n5x_swarming_rel/397363' --dump-json /tmp/tmpwP8ajA.json --expiration 3600 --io-timeout 1200 --hard-timeout 1500 --dimension device_os MMB29Q --dimension device_type bullhead --dimension os Android --dimension pool Chrome --dimension temp_band '<30' --named-cache swarming_module_cache_vpython .swarming_module_cache/vpython --tag build_is_experimental:false --tag buildername:android_n5x_swarming_rel --tag buildnumber:397363 --tag data:f34c255617c3d6a74802cc6a3b44e686a65b2eb7 --tag gerrit:https://chromium-review.googlesource.com/c/1000784/1 --tag master:tryserver.chromium.android --tag name:telemetry_perf_unittests --tag patch_project:chromium/src --tag project:chromium --tag purpose:CQ --tag purpose:pre-commit --tag slavename:slave657-c4 --tag 'stepname:telemetry_perf_unittests (with patch rerun) on Android' --user thakis@chromium.org --cipd-package '.swarming_module:infra/python/cpython/${platform}:version:2.7.14.chromium14' --cipd-package '.swarming_module:infra/tools/luci/logdog/butler/${platform}:git_revision:e1abc57be62d198b5c2f487bfb2fa2d2eb0e867c' --cipd-package '.swarming_module:infra/tools/luci/vpython-native/${platform}:git_revision:e1abc57be62d198b5c2f487bfb2fa2d2eb0e867c' --cipd-package '.swarming_module:infra/tools/luci/vpython/${platform}:git_revision:e1abc57be62d198b5c2f487bfb2fa2d2eb0e867c' --env-prefix PATH .swarming_module --env-prefix PATH .swarming_module/bin --env-prefix VPYTHON_VIRTUALENV_ROOT .swarming_module_cache/vpython --isolated 0b65cf924c2ceab2754ecf48e06a0633adc38b20  -- '--browser=android-chromium' '--device=android' '--isolated-script-test-output=${ISOLATED_OUTDIR}/output.json' '--isolated-script-test-perf-output=${ISOLATED_OUTDIR}/perftest-output.json'
Triggered task: telemetry_perf_unittests (with patch rerun)/Android/f34c255617/android_n5x_swarming_rel/397363
To collect results, use:
  swarming.py collect -S https://chromium-swarm.appspot.com --json /tmp/tmpwP8ajA.json
Or visit:
  https://chromium-swarm.appspot.com/user/task/3ccb089f0aabf310
  https://chromium-swarm.appspot.com/user/task/3ccb08a30d937610
  https://chromium-swarm.appspot.com/user/task/3ccb08a3d3c54610
  https://chromium-swarm.appspot.com/user/task/3ccb08a4e5f59110
  https://chromium-swarm.appspot.com/user/task/3ccb08a65472ca10
  https://chromium-swarm.appspot.com/user/task/3ccb08a9631c5910
  https://chromium-swarm.appspot.com/user/task/3ccb08aaa2615810
  https://chromium-swarm.appspot.com/user/task/3ccb08ac225d7310
  https://chromium-swarm.appspot.com/user/task/3ccb08ad526e8410
  https://chromium-swarm.appspot.com/user/task/3ccb08af70144f10
  https://chromium-swarm.appspot.com/user/task/3ccb08b651c86010
  https://chromium-swarm.appspot.com/user/task/3ccb08b7555d3d10
  https://chromium-swarm.appspot.com/user/task/3ccb08b950490d10
  https://chromium-swarm.appspot.com/user/task/3ccb08ba2cf6f310
  https://chromium-swarm.appspot.com/user/task/3ccb08bb1380e310
tools/swarming_client/swarming.py collect -S https://chromium-swarm.appspot.com --json /tmp/tmpwP8ajA.json > out2.txt

out2.txt now contains lots of

"""
  Usage: run_tests.py [test_name ...] [<options>]

  run_tests.py: error: No browser found of type android-chromium. Cannot run tests.
  Re-run with --browser=list to see available browser types.
"""

Looks like I'm holding things even more wrong than Hans :-(
Maybe it's important that the out dir is called 'Release'; I vaguely remember some of the python test runner stuff hardcoding it at some point. Trying again with that:

thakis@thakis:~/src/chrome/src$ mkdir out/Release
thakis@thakis:~/src/chrome/src$ cp out/gnand/args.gn out/Release/
thakis@thakis:~/src/chrome/src$ gn gen out/Release/
Done. Made 22159 targets from 1461 files in 9109ms
thakis@thakis:~/src/chrome/src$ tools/mb/mb.py isolate out/Release telemetry_perf_unittests
ninja -C out/Release telemetry_perf_unittests -j 960 -l 48
ninja: Entering directory `out/Release'
[55899/55899] STAMP obj/chrome/test/telemetry_perf_unittests.stamp
python tools/swarming_client/isolate.py check -i out/Release/telemetry_perf_unittests.isolate -s out/Release/telemetry_perf_unittests.isolated
thakis@thakis:~/src/chrome/src$ 
thakis@thakis:~/src/chrome/src$ tools/swarming_client/isolate.py archive -I https://isolateserver.appspot.com -i out/Release/telemetry_perf_unittests.isolate -s out/Release/telemetry_perf_unittests.isolated 
e34a413c1cd53a3342f609d87e1165a891fd6d4c  telemetry_perf_unittests
thakis@thakis:~/src/chrome/src$ 
thakis@thakis:~/src/chrome/src$ python tools/swarming_client/swarming.py trigger --swarming https://chromium-swarm.appspot.com --isolate-server https://isolateserver.appspot.com --priority 30 --shards 15 --task-name 'telemetry_perf_unittests (with patch rerun)/Android/f34c255617/android_n5x_swarming_rel/397363' --dump-json /tmp/tmpwP8ajA.json --expiration 3600 --io-timeout 1200 --hard-timeout 1500 --dimension device_os MMB29Q --dimension device_type bullhead --dimension os Android --dimension pool Chrome --dimension temp_band '<30' --named-cache swarming_module_cache_vpython .swarming_module_cache/vpython --tag build_is_experimental:false --tag buildername:android_n5x_swarming_rel --tag buildnumber:397363 --tag data:f34c255617c3d6a74802cc6a3b44e686a65b2eb7 --tag gerrit:https://chromium-review.googlesource.com/c/1000784/1 --tag master:tryserver.chromium.android --tag name:telemetry_perf_unittests --tag patch_project:chromium/src --tag project:chromium --tag purpose:CQ --tag purpose:pre-commit --tag slavename:slave657-c4 --tag 'stepname:telemetry_perf_unittests (with patch rerun) on Android' --user thakis@chromium.org --cipd-package '.swarming_module:infra/python/cpython/${platform}:version:2.7.14.chromium14' --cipd-package '.swarming_module:infra/tools/luci/logdog/butler/${platform}:git_revision:e1abc57be62d198b5c2f487bfb2fa2d2eb0e867c' --cipd-package '.swarming_module:infra/tools/luci/vpython-native/${platform}:git_revision:e1abc57be62d198b5c2f487bfb2fa2d2eb0e867c' --cipd-package '.swarming_module:infra/tools/luci/vpython/${platform}:git_revision:e1abc57be62d198b5c2f487bfb2fa2d2eb0e867c' --env-prefix PATH .swarming_module --env-prefix PATH .swarming_module/bin --env-prefix VPYTHON_VIRTUALENV_ROOT .swarming_module_cache/vpython --isolated e34a413c1cd53a3342f609d87e1165a891fd6d4c -- '--browser=android-chromium' '--device=android' '--isolated-script-test-output=${ISOLATED_OUTDIR}/output.json' '--isolated-script-test-perf-output=${ISOLATED_OUTDIR}/perftest-output.json'
Triggered task: telemetry_perf_unittests (with patch rerun)/Android/f34c255617/android_n5x_swarming_rel/397363
To collect results, use:
  swarming.py collect -S https://chromium-swarm.appspot.com --json /tmp/tmpwP8ajA.json
Or visit:
  https://chromium-swarm.appspot.com/user/task/3ccb3929f30cd710
  https://chromium-swarm.appspot.com/user/task/3ccb392d1d6d0510
  https://chromium-swarm.appspot.com/user/task/3ccb393035901810
  https://chromium-swarm.appspot.com/user/task/3ccb39331a9ffe10
  https://chromium-swarm.appspot.com/user/task/3ccb39357b996710
  https://chromium-swarm.appspot.com/user/task/3ccb393967c1b510
  https://chromium-swarm.appspot.com/user/task/3ccb393b2f6bc010
  https://chromium-swarm.appspot.com/user/task/3ccb393d9fbf1e10
  https://chromium-swarm.appspot.com/user/task/3ccb39400a044410
  https://chromium-swarm.appspot.com/user/task/3ccb3943225a3610
  https://chromium-swarm.appspot.com/user/task/3ccb3945e72b1e10
  https://chromium-swarm.appspot.com/user/task/3ccb39480ce84e10
  https://chromium-swarm.appspot.com/user/task/3ccb3949caa4fd10
  https://chromium-swarm.appspot.com/user/task/3ccb394cacca8010
  https://chromium-swarm.appspot.com/user/task/3ccb394e5db84a10
Success!


thakis@thakis:~/src/chrome/src$ tools/swarming_client/swarming.py collect -S https://chromium-swarm.appspot.com --json /tmp/tmpwP8ajA.json > foo3.txt


thakis@thakis:~/src/chrome/src$ grep 'beginning of crash' foo3.txt 
        --------- beginning of crash
        --------- beginning of crash
        --------- beginning of crash
        --------- beginning of crash
        --------- beginning of crash
        --------- beginning of crash
        --------- beginning of crash
        --------- beginning of crash
With the clang roll reverted locally it doesn't repro:



thakis@thakis:~/src/chrome/src$ git checkout -f HEAD
tools/clang/scripts/update.pyYour branch is up to date with 'origin/master'.
thakis@thakis:~/src/chrome/src$ tools/clang/scripts/update.py
Downloading https://commondatastorage.googleapis.com/chromium-browser-clang/Linux_x64/clang-328716-2.tgz .......... Done.
Downloading https://commondatastorage.googleapis.com/chromium-browser-clang/Win/clang-328716-2.tgz .......... Done.
thakis@thakis:~/src/chrome/src$ tools/mb/mb.py isolate out/Release telemetry_perf_unittests
ninja -C out/Release telemetry_perf_unittests -j 960 -l 48
ninja: Entering directory `out/Release'
[1/1] Regenerating ninja files
[47028/47028] STAMP obj/android_webview/system_webview_apk.stamp
python tools/swarming_client/isolate.py check -i out/Release/telemetry_perf_unittests.isolate -s out/Release/telemetry_perf_unittests.isolated
thakis@thakis:~/src/chrome/src$ tools/swarming_client/isolate.py archive -I https://isolateserver.appspot.com -i out/Release/telemetry_perf_unittests.isolate -s out/Release/telemetry_perf_unittests.isolated 
59f7906786f7bd205bbf5c7c58fe5a8750319f51  telemetry_perf_unittests
thakis@thakis:~/src/chrome/src$  python tools/swarming_client/swarming.py trigger --swarming https://chromium-swarm.appspot.com --isolate-server https://isolateserver.appspot.com --priority 30 --shards 15 --task-name 'telemetry_perf_unittests (with patch rerun)/Android/f34c255617/android_n5x_swarming_rel/397363' --dump-json /tmp/tmpwP8ajA.json --expiration 3600 --io-timeout 1200 --hard-timeout 1500 --dimension device_os MMB29Q --dimension device_type bullhead --dimension os Android --dimension pool Chrome --dimension temp_band '<30' --named-cache swarming_module_cache_vpython .swarming_module_cache/vpython --tag build_is_experimental:false --tag buildername:android_n5x_swarming_rel --tag buildnumber:397363 --tag data:f34c255617c3d6a74802cc6a3b44e686a65b2eb7 --tag gerrit:https://chromium-review.googlesource.com/c/1000784/1 --tag master:tryserver.chromium.android --tag name:telemetry_perf_unittests --tag patch_project:chromium/src --tag project:chromium --tag purpose:CQ --tag purpose:pre-commit --tag slavename:slave657-c4 --tag 'stepname:telemetry_perf_unittests (with patch rerun) on Android' --user thakis@chromium.org --cipd-package '.swarming_module:infra/python/cpython/${platform}:version:2.7.14.chromium14' --cipd-package '.swarming_module:infra/tools/luci/logdog/butler/${platform}:git_revision:e1abc57be62d198b5c2f487bfb2fa2d2eb0e867c' --cipd-package '.swarming_module:infra/tools/luci/vpython-native/${platform}:git_revision:e1abc57be62d198b5c2f487bfb2fa2d2eb0e867c' --cipd-package '.swarming_module:infra/tools/luci/vpython/${platform}:git_revision:e1abc57be62d198b5c2f487bfb2fa2d2eb0e867c' --env-prefix PATH .swarming_module --env-prefix PATH .swarming_module/bin --env-prefix VPYTHON_VIRTUALENV_ROOT .swarming_module_cache/vpython --isolated 59f7906786f7bd205bbf5c7c58fe5a8750319f51   -- '--browser=android-chromium' '--device=android' '--isolated-script-test-output=${ISOLATED_OUTDIR}/output.json' '--isolated-script-test-perf-output=${ISOLATED_OUTDIR}/perftest-output.json'
Triggered task: telemetry_perf_unittests (with patch rerun)/Android/f34c255617/android_n5x_swarming_rel/397363
To collect results, use:
  swarming.py collect -S https://chromium-swarm.appspot.com --json /tmp/tmpwP8ajA.json
Or visit:
  https://chromium-swarm.appspot.com/user/task/3ccb5c96eda0bc10
  https://chromium-swarm.appspot.com/user/task/3ccb5c98bcaf4110
  https://chromium-swarm.appspot.com/user/task/3ccb5c9adfe35310
  https://chromium-swarm.appspot.com/user/task/3ccb5c9c5142d610
  https://chromium-swarm.appspot.com/user/task/3ccb5c9edd31f510
  https://chromium-swarm.appspot.com/user/task/3ccb5ca060bcbd10
  https://chromium-swarm.appspot.com/user/task/3ccb5ca1a5484b10
  https://chromium-swarm.appspot.com/user/task/3ccb5ca2f48a2910
  https://chromium-swarm.appspot.com/user/task/3ccb5ca43a8c4710
  https://chromium-swarm.appspot.com/user/task/3ccb5ca537fa1710
  https://chromium-swarm.appspot.com/user/task/3ccb5ca63c88a510
  https://chromium-swarm.appspot.com/user/task/3ccb5ca79677e910
  https://chromium-swarm.appspot.com/user/task/3ccb5ca878dfc310
  https://chromium-swarm.appspot.com/user/task/3ccb5ca95c2ed610
  https://chromium-swarm.appspot.com/user/task/3ccb5caa93468510
thakis@thakis:~/src/chrome/src$ tools/swarming_client/swarming.py collect -S https://chromium-swarm.appspot.com --json /tmp/tmpwP8ajA.json > foo4.txt
thakis@thakis:~/src/chrome/src$  grep 'beginning of crash' foo4.txt

Comment 29 by h...@chromium.org, Apr 11 2018

Status: Started (was: Available)
> Maybe it's important that the out dir is called 'Release'; I vaguely remember some of the python test runner stuff hardcoding it at some point.

Yay, that's it!! Thanks Nico!

I also got the "No browser found of type android-chromium." message yesterday, so I tried dropping the --browser=android-chromium flag, which made the test run but presumably using some other binary because it didn't crash.

But building in out/Release instead of out/release  makes it repro. (*grumbles*)


Copying my steps for completeness:

$ gn gen out/Release
$ tools/mb/mb.py isolate //out/Release telemetry_perf_unittests 
$ tools/swarming_client/isolate.py archive -I https://isolateserver.appspot.com -i out/Release/telemetry_perf_unittests.isolate -s out/Release/telemetry_perf_unittests.isolated --verbose
$ export hash=63e2732f41066de9853ca2930c5676e38752c597

$ tools/swarming_client/swarming.py trigger --swarming https://chromium-swarm.appspot.com --isolate-server https://isolateserver.appspot.com  --dimension device_os MMB29Q --dimension device_type bullhead --dimension os Android --dimension pool Chrome --named-cache swarming_module_cache_vpython .swarming_module_cache/vpython  --cipd-package '.swarming_module:infra/python/cpython/${platform}:version:2.7.14.chromium14' --cipd-package '.swarming_module:infra/tools/luci/logdog/butler/${platform}:git_revision:e1abc57be62d198b5c2f487bfb2fa2d2eb0e867c' --cipd-package '.swarming_module:infra/tools/luci/vpython-native/${platform}:git_revision:e1abc57be62d198b5c2f487bfb2fa2d2eb0e867c' --cipd-package '.swarming_module:infra/tools/luci/vpython/${platform}:git_revision:e1abc57be62d198b5c2f487bfb2fa2d2eb0e867c' --env-prefix PATH .swarming_module --env-prefix PATH .swarming_module/bin --env-prefix VPYTHON_VIRTUALENV_ROOT .swarming_module_cache/vpython --isolated $hash -- '--browser=android-chromium' '--device=android' '--isolated-script-test-output=${ISOLATED_OUTDIR}/output.json' '--isolated-script-test-perf-output=${ISOLATED_OUTDIR}/perftest-output.json' --isolated-script-test-filter=benchmarks.system_health_smoke_test.SystemHealthBenchmarkSmokeTest.system_health.memory_mobile.browse:shopping:avito

(The results are at https://chromium-swarm.appspot.com/user/task/3ccc7c668b19ab10)



Let's bisect this!
Cc: kbr@chromium.org
thakis@, this is epic work that you did to figure out how to trigger test on swarming with a local build! 

Do you mind documenting this since it will be useful to many other folks
Sure, can do. What would be a good place for documenting this?

Also, I wonder if something (maybe mb?) should shout at you when you try to run telemetry tests with an out dir that isn't 'out/Release'.
thakis@: https://chromium.googlesource.com/chromium/src/+/master/docs/speed/benchmark/ would be a good place to add the documentation

*I leave mb question to Dirk to answer

Comment 33 by h...@chromium.org, Apr 11 2018

Still bisecting..

(Note to self:)
$ gn clean out/Release && gn gen out/Release && tools/mb/mb.py isolate //out/Release telemetry_perf_unittests
$ tools/swarming_client/isolate.py archive -I https://isolateserver.appspot.com -i out/Release/telemetry_perf_unittests.isolate -s out/Release/telemetry_perf_unittests.isolated
$ export hash=b43658bbf7ae084b916ae500c4ad24910daeb3bf
(trigger as above)

Comment 34 by h...@chromium.org, Apr 11 2018

Still bisecting, but the range is small now

I don't know what "mir-canon" is, but besides that only the dag-combine seems relevant.

$ git bisect visualize --stat

commit b36d3507b12315e20f32a2723d6252379d948e55
Author: Nico Weber <nicolasweber@gmx.de>
Date:   Sat Mar 31 18:26:25 2018 +0000

    Revert r328845, it caused  crbug.com/827810 .

 clang/include/clang/AST/Mangle.h        |  4 +-
 clang/include/clang/AST/VTableBuilder.h | 67 ++++++++++++++++-----------------
 clang/lib/AST/MicrosoftMangle.cpp       | 24 +++++++-----
 clang/lib/AST/VTableBuilder.cpp         | 11 ++++--
 clang/lib/CodeGen/CGDebugInfo.cpp       |  2 +-
 clang/lib/CodeGen/MicrosoftCXXABI.cpp   | 24 ++++++------
 6 files changed, 70 insertions(+), 62 deletions(-)

commit 671677e75d973c90fa74506cf6deb3aec4811bed
Author: Sanjay Patel <spatel@rotateright.com>
Date:   Sat Mar 31 17:55:44 2018 +0000

    [DAGCombine] (float)((int) f) --> ftrunc (PR36617)
    
    fptosi / fptoui round towards zero, and that's the same behavior as ISD::FTRUNC,
    so replace a pair of casts with the equivalent node. We don't have to account for
    special cases (NaN, INF) because out-of-range casts are undefined.
    
    Differential Revision: https://reviews.llvm.org/D44909

 llvm/include/llvm/CodeGen/ISDOpcodes.h             |   3 +-
 llvm/lib/CodeGen/SelectionDAG/DAGCombiner.cpp      |  18 ++
 llvm/test/CodeGen/AArch64/ftrunc.ll                |  12 +-
 llvm/test/CodeGen/ARM/ftrunc.ll                    |  42 +++
 llvm/test/CodeGen/PowerPC/fp-int128-fp-combine.ll  |  13 +-
 llvm/test/CodeGen/PowerPC/fp-to-int-to-fp.ll       |  12 +-
 llvm/test/CodeGen/PowerPC/ftrunc-vec.ll            |  12 +-
 llvm/test/CodeGen/PowerPC/no-extra-fp-conv-ldst.ll |  24 +-
 llvm/test/CodeGen/X86/2011-10-19-widen_vselect.ll  |   6 +-
 llvm/test/CodeGen/X86/ftrunc.ll                    | 292 ++-------------------
 10 files changed, 103 insertions(+), 331 deletions(-)

commit 95240e8571408ce3b9ddf718c0522175c7303818
Author: Lang Hames <lhames@gmail.com>
Date:   Sat Mar 31 16:01:01 2018 +0000

    [llvm-rtdyld] Fix the InputFileList cl::opt description: it accepts multiple
    input files.

 llvm/tools/llvm-rtdyld/llvm-rtdyld.cpp | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

commit 8efed426cdf06ea51c9d6998e460bbcc92acfc47
Author: Henry Wong <movietravelcode@outlook.com>
Date:   Sat Mar 31 12:46:46 2018 +0000

    [analyzer] Unroll the loop when it has a unsigned counter.
    
    Summary:
    The original implementation in the `LoopUnrolling.cpp` didn't consider the case where the counter is unsigned. This case is only handled in `simpleCondition()`, but this is not enough, we also need to deal with the unsinged counter with the counter initialization.
    
    Since `IntegerLiteral` is `signed`, there is a `ImplicitCastExpr<IntegralCast>` in `unsigned counter = IntergerLiteral`. This patch add the `ignoringParenImpCasts()` in the `IntegerLiteral` matcher.
    
    Reviewers: szepet, a.sidorin, NoQ, george.karpenkov
    
    Reviewed By: szepet, george.karpenkov
    
    Subscribers: xazax.hun, rnkovacs, cfe-commits, MTC
    
    Differential Revision: https://reviews.llvm.org/D45086

 clang/lib/StaticAnalyzer/Core/LoopUnrolling.cpp | 16 +++++++++-------
 clang/test/Analysis/loop-unrolling.cpp          | 23 +++++++++++++++++++++++
 2 files changed, 32 insertions(+), 7 deletions(-)

commit daff3829d9ec0fcac8a33a006ae93f789f2f4c63
Author: Simon Pilgrim <llvm-dev@redking.me.uk>
Date:   Sat Mar 31 09:15:54 2018 +0000

    [X86][Btver2] Add MMX_PSHUFB to the JWritePSHUFB InstRW entries

 llvm/lib/Target/X86/X86ScheduleBtVer2.td | 4 ++--
 llvm/test/CodeGen/X86/mmx-schedule.ll    | 4 ++--
 2 files changed, 4 insertions(+), 4 deletions(-)

commit 3420206dfcc659388cdb227d8a7a3ff4c92c5c92
Author: Simon Pilgrim <llvm-dev@redking.me.uk>
Date:   Sat Mar 31 09:14:14 2018 +0000

    Fix trailing whitespace. NFCI.

 llvm/lib/Target/X86/X86ISelLowering.cpp | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

commit 8ca4f87b299ebaf039c6bbc55ab2790cf7eb5235
Author: Benjamin Kramer <benny.kra@googlemail.com>
Date:   Sat Mar 31 07:41:25 2018 +0000

    Unbreak the build of the go bindings after r328839.

 llvm/bindings/go/llvm/transforms_scalar.go | 1 +
 1 file changed, 1 insertion(+)

commit 417b2fe1652fdc96777fc2d16227f5d044cb0985
Author: Puyan Lotfi <puyan@puyan.org>
Date:   Sat Mar 31 05:48:51 2018 +0000

    [MIR-Canon] Adding support for local idempotent instruction hoisting.

 llvm/lib/CodeGen/MIRCanonicalizerPass.cpp          | 121 ++++++++++++++++++++-
 .../CodeGen/MIR/AArch64/mirCanonIdempotent.mir     | 116 ++++++++++++++++++++
 2 files changed, 232 insertions(+), 5 deletions(-)

commit 28cacab8493d4ae24e7471352d7d12c6cf556b3e
Author: Craig Topper <craig.topper@intel.com>
Date:   Sat Mar 31 04:54:32 2018 +0000

    [X86] Add SchedRW for PMULLD
    
    Summary:
    It seems many CPUs don't implement this instruction as well as the other vector multiplies. Often using a multi uop flow. Silvermont in particular has a 7 uop flow with 11 cycle throughput. Sandy Bridge implements it as a single uop with 5 cycle latency and 1 cycle throughput. But Haswell and later use 2 uops with 10 cycle latency and 2 cycle throughput.
    
    This patch adds a new X86SchedWritePair we can use to tag this instruction separately. I've provided correct information for Silvermont, Btver2, and Sandy Bridge. I've removed the InstRWs for SandyBridge. I've left Haswell/Broadwell/Skylake InstRWs in place because I wasn't sure how to account for the different load latency between 128 and 256 bits. I also left Znver1 InstRWs in place because the existing values don't match Agner's spreadsheet.
    
    I also left a FIXME in the SandyBridge model because it being used for the "generic" model is too optimistic for the 256/512-bit versions since those are multiple uops on all known CPUs.
    
    Reviewers: RKSimon, GGanesh, courbet
    
    Reviewed By: RKSimon
    
    Subscribers: gchatelet, gbedwell, andreadb, llvm-commits
    
    Differential Revision: https://reviews.llvm.org/D44972

 llvm/lib/Target/X86/X86InstrAVX512.td              |  2 +-
 llvm/lib/Target/X86/X86InstrSSE.td                 |  2 +-
 llvm/lib/Target/X86/X86SchedBroadwell.td           | 15 +-------
 llvm/lib/Target/X86/X86SchedHaswell.td             | 15 +-------
 llvm/lib/Target/X86/X86SchedSandyBridge.td         |  3 +-
 llvm/lib/Target/X86/X86SchedSkylakeClient.td       | 15 +-------
 llvm/lib/Target/X86/X86SchedSkylakeServer.td       |  1 +
 llvm/lib/Target/X86/X86Schedule.td                 |  1 +
 llvm/lib/Target/X86/X86ScheduleBtVer2.td           |  1 +
 llvm/lib/Target/X86/X86ScheduleSLM.td              |  1 +
 llvm/lib/Target/X86/X86ScheduleZnver1.td           |  1 +
 llvm/test/CodeGen/X86/avx2-schedule.ll             |  2 +-
 llvm/test/CodeGen/X86/slow-pmulld.ll               | 42 ++++++++++----------
 llvm/test/CodeGen/X86/sse41-schedule.ll            | 12 +++---
 llvm/test/tools/llvm-mca/X86/BtVer2/pipes-fpu.s    | 45 +++++++++++-----------
 .../tools/llvm-mca/X86/BtVer2/resources-avx1.s     |  8 ++--
 .../tools/llvm-mca/X86/BtVer2/resources-sse41.s    |  4 +-
 17 files changed, 66 insertions(+), 104 deletions(-)

commit cb1028a1c37e7cf5fcf1e727ae6c68d269987b10
Author: George Karpenkov <ekarpenkov@apple.com>
Date:   Sat Mar 31 02:17:15 2018 +0000

    [analyzer] Hopefully fix the ARM buildbot.

 clang/test/Analysis/bstring.c |  9 +++++++++
 clang/test/Analysis/string.c  | 10 ----------
 2 files changed, 9 insertions(+), 10 deletions(-)

commit b012cb268379a54a3b6cb9b4504c848334cf993e
Author: George Karpenkov <ekarpenkov@apple.com>
Date:   Sat Mar 31 01:20:08 2018 +0000

    [analyzer] Fix assertion crash in CStringChecker
    
    An offset might be unknown.
    
    rdar://39054939
    
    Differential Revision: https://reviews.llvm.org/D45115

 clang/lib/StaticAnalyzer/Checkers/CStringChecker.cpp | 13 ++++++++-----
 clang/test/Analysis/string.c                         | 11 +++++++++++
 2 files changed, 19 insertions(+), 5 deletions(-)

commit 944d0c4eacb83ec237501426839a13be824e6a0c
Author: George Karpenkov <ekarpenkov@apple.com>
Date:   Sat Mar 31 01:20:07 2018 +0000

    [analyzer] Cache offset computation for MemRegion
    
    Achieves almost a 200% speedup on the example where the performance of
    visitors was problematic.
    
    Performance on sqlite3 is unaffected.
    
    rdar://38818362
    
    Differential Revision: https://reviews.llvm.org/D45113

 .../StaticAnalyzer/Core/PathSensitive/MemRegion.h  |  1 +
 clang/lib/StaticAnalyzer/Core/MemRegion.cpp        | 67 ++++++++++++----------
 2 files changed, 37 insertions(+), 31 deletions(-)

commit 28f0bedebdea3302630aa3ad003246dbdb612122
Author: George Karpenkov <ekarpenkov@apple.com>
Date:   Sat Mar 31 01:20:06 2018 +0000

    [analyzer] Fix liveness calculation for C++17 structured bindings
    
    C++ structured bindings for non-tuple-types are defined in a peculiar
    way, where the resulting declaration is not a VarDecl, but a
    BindingDecl.
    That means a lot of existing machinery stops working.
    
    rdar://36912381
    
    Differential Revision: https://reviews.llvm.org/D44956

 .../clang/Analysis/Analyses/LiveVariables.h        |   9 +-
 clang/lib/Analysis/LiveVariables.cpp               |  83 ++++++++++----
 clang/test/Analysis/live-bindings-test.cpp         | 124 +++++++++++++++++++++
 3 files changed, 189 insertions(+), 27 deletions(-)

commit 3729e9d8558f491d97df7cad80529cd25eb34f33
Author: Teresa Johnson <tejohnson@google.com>
Date:   Sat Mar 31 00:18:08 2018 +0000

    [ThinLTO] Add an option to force summary call edges cold for debugging
    
    Summary:
    Useful to selectively disable importing into specific modules for
    debugging/triaging/workarounds.
    
    Reviewers: eraman
    
    Subscribers: inglorion, llvm-commits
    
    Differential Revision: https://reviews.llvm.org/D45062

 llvm/include/llvm/IR/ModuleSummaryIndex.h          |  7 ++++
 llvm/lib/Analysis/ModuleSummaryAnalysis.cpp        | 19 ++++++++++-
 llvm/lib/Bitcode/Writer/BitcodeWriter.cpp          |  6 +++-
 .../FunctionImport/Inputs/funcimport_forcecold.ll  |  4 +++
 .../FunctionImport/funcimport_forcecold.ll         | 36 +++++++++++++++++++++
 .../funcimport_forcecold_samplepgo.ll              | 37 ++++++++++++++++++++++
 6 files changed, 107 insertions(+), 2 deletions(-)

commit f26055f65d16c10cc880dbf237c7053d25584861
Author: Fangrui Song <maskray@google.com>
Date:   Fri Mar 30 23:13:00 2018 +0000

    [ELF] Simplify read32. NFC

 lld/ELF/EhFrame.cpp | 5 ++---
 1 file changed, 2 insertions(+), 3 deletions(-)

commit a0e1873f80acf1befb559738a0e1c31916fb2512
Author: Fangrui Song <maskray@google.com>
Date:   Fri Mar 30 22:22:31 2018 +0000

    Fix a bunch of typoes. NFC

 llvm/lib/Analysis/TargetLibraryInfo.cpp               | 2 +-
 llvm/lib/CodeGen/SelectionDAG/SelectionDAGBuilder.cpp | 2 +-
 llvm/lib/CodeGen/SelectionDAG/SelectionDAGISel.cpp    | 2 +-
 llvm/lib/CodeGen/StackColoring.cpp                    | 2 +-
 llvm/lib/Target/BPF/BPFISelDAGToDAG.cpp               | 2 +-
 llvm/lib/Target/Hexagon/HexagonISelDAGToDAGHVX.cpp    | 2 +-
 llvm/lib/Target/Hexagon/HexagonISelLowering.cpp       | 2 +-
 llvm/lib/Target/Hexagon/HexagonNewValueJump.cpp       | 8 ++++----
 llvm/lib/Target/Mips/MipsInstrInfo.cpp                | 2 +-
 llvm/lib/Target/X86/MCTargetDesc/X86AsmBackend.cpp    | 3 +--
 llvm/lib/Transforms/Scalar/ADCE.cpp                   | 4 ++--
 llvm/lib/Transforms/Scalar/LICM.cpp                   | 2 +-
 llvm/utils/TableGen/CodeGenDAGPatterns.cpp            | 2 +-
 llvm/utils/TableGen/X86FoldTablesEmitter.cpp          | 2 +-
 14 files changed, 18 insertions(+), 19 deletions(-)

Comment 35 by h...@chromium.org, Apr 11 2018

Bisection points to r328921:

Author: spatel
Date: Sat Mar 31 10:55:44 2018
New Revision: 328921

URL: http://llvm.org/viewvc/llvm-project?rev=328921&view=rev
Log:
[DAGCombine] (float)((int) f) --> ftrunc (PR36617)

fptosi / fptoui round towards zero, and that's the same behavior as ISD::FTRUNC, 
so replace a pair of casts with the equivalent node. We don't have to account for 
special cases (NaN, INF) because out-of-range casts are undefined.

Differential Revision: https://reviews.llvm.org/D44909
Left an "any idea what might be happening" type of comment over there.
(1) mb shouldn't shout at you if you don't use a directory named 'Release' or 'Debug'. Rather, it's a bug in any test harness that such a directory might be required.

(2) The right way to handle this is to make `mb run --swarmed` make this as easy as possible to reproduce a test step. It has about half the plumbing it needs now, but it still needs the code to look up the test step configuration in the //testing/buildbot/*.json files (and the correct swarming dimensions).
Cc: sullivan@chromium.org
Where "Release" is mentioned in telemetry: https://cs.chromium.org/search/?q=%5CbRelease+case:yes+file:telemetry&sq=package:chromium&type=cs

So primarily catapult. There: https://cs.chromium.org/search/?q=%5CbRelease%5Cb+case:yes+file:catapult&sq=package:chromium&type=cs

catapult/OWNERS are nednguyen and sullivan. Does either of you two have a sense for how hard it is to make catapult not require a build dir that's called 'Release'? (See comments 37 and 26) How long would it take to change this?

Comment 39 by h...@chromium.org, Apr 12 2018

Owner: h...@chromium.org
> Left an "any idea what might be happening" type of comment over there.

On llvm-commits? I didn't see anything.


I just found that this revision also caused problems internally (b/77585750). Hopefully we can get some kind of diagnosis that's good enough for a revert today.

Comment 40 by h...@chromium.org, Apr 12 2018

Attaching the list of changed files after the culprit revision.

Unfortunately it's a pretty long list, but this one stands out as being relevant to the DCHECK:

./obj/third_party/blink/renderer/modules/accessibility/accessibility/ax_layout_object.o
changed_files.txt
12.4 KB View Download
#38: Telemetry tests can be made to not depend on build dir that's called 'Release' by modifying the test commandline. 

I believe you can do change its command line flag to:

--browser=exact ----browser-executable=<path to apk>

What output directory are you using?

+Juan: since if there are browser apks in non Release/ directory, we should also modify browser_finder to look for those.

Comment 43 by h...@chromium.org, Apr 12 2018

It doesn't seem to be ax_layout_object.o. The function that changes in that file is _ZNK5blink14AXLayoutObject12ImageDataUrlERKNS_7IntSizeE and marking it optnone didn't help.

Also looking at the diff in that file manually, the change seemed valid.

Unfortunately this means the problem could be in a lot of places..


I also split the DCHECK that's failing: the container.GetNode() part returns nullptr.

Comment 45 by h...@chromium.org, Apr 12 2018

Copying clang from the 328920 package above (the good version) into third_party/llvm-build/Release+Asserts/bin/, removing some of the object files from the list in #40 and re-building should allow for figuring out what file (hopefully it's just one) that's broken.

So far it appears to be in V8 (which also makes sense given the internal bug). Re-building these files with the good Clang makes the test pass:

./obj/v8/src/inspector/inspector/Protocol.o
./obj/v8/src/inspector/inspector/v8-value-utils.o
./obj/v8/v8_base/api.o
./obj/v8/v8_base/assembler.o
./obj/v8/v8_base/builtins-dataview.o
./obj/v8/v8_base/builtins-sharedarraybuffer.o
./obj/v8/v8_base/builtins-string.o
./obj/v8/v8_base/conversions.o
./obj/v8/v8_base/deserializer.o
./obj/v8/v8_base/elements.o
./obj/v8/v8_base/escape-analysis.o
./obj/v8/v8_base/external-reference.o
./obj/v8/v8_base/ic.o
./obj/v8/v8_base/instruction-selector.o
./obj/v8/v8_base/json-stringifier.o
./obj/v8/v8_base/machine-operator-reducer.o
./obj/v8/v8_base/objects.o
./obj/v8/v8_base/parser.o
./obj/v8/v8_base/representation-change.o
./obj/v8/v8_base/runtime-array.o
./obj/v8/v8_base/runtime-atomics.o
./obj/v8/v8_base/runtime-collections.o
./obj/v8/v8_base/runtime-debug.o
./obj/v8/v8_base/runtime-liveedit.o
./obj/v8/v8_base/runtime-numbers.o
./obj/v8/v8_base/runtime-regexp.o
./obj/v8/v8_base/runtime-strings.o
./obj/v8/v8_base/runtime-test.o
./obj/v8/v8_base/runtime-wasm.o
./obj/v8/v8_base/serializer.o
./obj/v8/v8_base/simplified-lowering.o
./obj/v8/v8_base/simplified-operator-reducer.o
./obj/v8/v8_base/types.o
./obj/v8/v8_base/value-serializer.o
./obj/v8/v8_base/wasm-interpreter.o
./obj/v8/v8_libbase/ieee754.o

Let's shrink that list...
Cc: cbruni@chromium.org hpayer@chromium.org u...@chromium.org
+v8 folks who may be interested in this investigation.

See #45

Comment 47 by h...@chromium.org, Apr 12 2018

v8 folks: This is tracking a regression in Clang. There's nothing to suggest there's a problem with v8, only that it's v8 that's getting misscompiled.

Bisection points to ./obj/v8/v8_base/ic.o

Comment 48 by h...@chromium.org, Apr 12 2018

And the diff is small..

$ diff -u  <(./third_party/android_tools/ndk/toolchains/aarch64-linux-android-4.9/prebuilt/linux-x86_64/bin/aarch64-linux-android-objdump -d out/Release.good/./obj/v8/v8_base/ic.o ) <(./third_party/android_tools/ndk/toolchains/aarch64-linux-android-4.9/prebuilt/linux-x86_64/bin/aarch64-linux-android-objdump -d out/Release.bad/./obj/v8/v8_base/ic.o )
--- /dev/fd/63  2018-04-12 15:59:04.269197920 +0200
+++ /dev/fd/62  2018-04-12 15:59:04.269197920 +0200
@@ -1,5 +1,5 @@
 
-out/Release.good/./obj/v8/v8_base/ic.o:     file format elf64-littleaarch64
+out/Release.bad/./obj/v8/v8_base/ic.o:     file format elf64-littleaarch64
 
 
 Disassembly of section .text._ZN2v88internal2IC23TransitionMarkFromStateENS0_16InlineCacheStateE:
@@ -8556,10 +8556,10 @@
   5c:  34000060        cbz     w0, 68 <_ZN2v88internalL13TryConvertKeyENS0_6HandleINS0_6ObjectEEEPNS0_7IsolateE+0x68>
   60:  91278274        add     x20, x19, #0x9e0
   64:  1400004b        b       190 <_ZN2v88internalL13TryConvertKeyENS0_6HandleINS0_6ObjectEEEPNS0_7IsolateE+0x190>
-  68:  1e780115        fcvtzs  w21, d8
-  6c:  1e6202a0        scvtf   d0, w21
-  70:  1e602100        fcmp    d8, d0
-  74:  540008e1        b.ne    190 <_ZN2v88internalL13TryConvertKeyENS0_6HandleINS0_6ObjectEEEPNS0_7IsolateE+0x190>
+  68:  1e65c100        frintz  d0, d8
+  6c:  1e602100        fcmp    d8, d0
+  70:  54000901        b.ne    190 <_ZN2v88internalL13TryConvertKeyENS0_6HandleINS0_6ObjectEEEPNS0_7IsolateE+0x190>
+  74:  1e780115        fcvtzs  w21, d8
   78:  320103e8        orr     w8, wzr, #0x80000000
   7c:  8b35c108        add     x8, x8, w21, sxtw
   80:  d360fd08        lsr     x8, x8, #32


This is the source:

static Handle<Object> TryConvertKey(Handle<Object> key, Isolate* isolate) {
  // This helper implements a few common fast cases for converting
  // non-smi keys of keyed loads/stores to a smi or a string.
  if (key->IsHeapNumber()) {
    double value = Handle<HeapNumber>::cast(key)->value();
    if (std::isnan(value)) {
      key = isolate->factory()->NaN_string();
    } else {
      int int_value = FastD2I(value);
      if (value == int_value && Smi::IsValid(int_value)) {

What's happening is that the conversion from int and back (fcvtzs scvtf) before the comparison is folded into just rounding the float (frintz).


I'm not sure this code is sound:

      int int_value = FastD2I(value);
      if (value == int_value

The code attempts to check that casting the double to int preserves the value. But if the value doesn't fit into an int, the cast has unspecified result, which is the reasoning the compiler relies on to fold the conversion away.
Cc: jkummerow@chromium.org
nice digging, this is probably not the last time V8 is fighting against the compiler :)

+jkummerow@ who has been working on correcter-er V8 ;)

Comment 50 by h...@chromium.org, Apr 12 2018

V8 has more instances of this by the way, e.g.

bool Object::ToInt32(int32_t* value) {
  if (IsSmi()) {
    *value = Smi::ToInt(this);
    return true;
  }
  if (IsHeapNumber()) {
    double num = HeapNumber::cast(this)->value();
    if (FastI2D(FastD2I(num)) == num) {
      *value = FastD2I(num);
      return true;
    }
  }
  return false;
}

Comment 51 by h...@chromium.org, Apr 12 2018

conversions-inl.h has

bool DoubleToSmiInteger(double value, int* smi_int_value) {
  if (IsMinusZero(value)) return false;
  int i = FastD2IChecked(value);
  if (value != i || !Smi::IsValid(i)) return false;
  *smi_int_value = i;
  return true;
}

Which looks like a more principled way of what TryConvertKey is doing.
The upstream llvm change was reverted in https://reviews.llvm.org/rL329920
primiano got the crash stack symbolized:


logging::LogMessage::~LogMessage()
 
blink::AXPosition::CreatePositionInTextObject(blink::AXObject const&, int, blink::TextAffinity)
 
blink::AXInlineTextBox::GetWordBoundaries(WTF::Vector<blink::AXRange, 0ul, WTF::PartitionAllocator>&) const
 
blink::WebAXObject::GetWordBoundaries(blink::WebVector<int>&, blink::WebVector<int>&) const
 
content::BlinkAXTreeSource::SerializeNode(blink::WebAXObject, content::AXContentNodeData*) const
 
ui::AXTreeSerializer<blink::WebAXObject, content::AXContentNodeData, content::AXContentTreeData>::SerializeChangedNodes(blink::WebAXObject, ui::AXTreeUpdateBase<content::AXContentNodeData, content::AXContentTreeData>*)
 
ui::AXTreeSerializer<blink::WebAXObject, content::AXContentNodeData, content::AXContentTreeData>::SerializeChangedNodes(blink::WebAXObject, ui::AXTreeUpdateBase<content::AXContentNodeData, content::AXContentTreeData>*)
 
ui::AXTreeSerializer<blink::WebAXObject, content::AXContentNodeData, content::AXContentTreeData>::SerializeChangedNodes(blink::WebAXObject, ui::AXTreeUpdateBase<content::AXContentNodeData, content::AXContentTreeData>*)
 
ui::AXTreeSerializer<blink::WebAXObject, content::AXContentNodeData, content::AXContentTreeData>::SerializeChangedNodes(blink::WebAXObject, ui::AXTreeUpdateBase<content::AXContentNodeData, content::AXContentTreeData>*)
 
ui::AXTreeSerializer<blink::WebAXObject, content::AXContentNodeData, content::AXContentTreeData>::SerializeChangedNodes(blink::WebAXObject, ui::AXTreeUpdateBase<content::AXContentNodeData, content::AXContentTreeData>*)
 
ui::AXTreeSerializer<blink::WebAXObject, content::AXContentNodeData, content::AXContentTreeData>::SerializeChangedNodes(blink::WebAXObject, ui::AXTreeUpdateBase<content::AXContentNodeData, content::AXContentTreeData>*)
 
ui::AXTreeSerializer<blink::WebAXObject, content::AXContentNodeData, content::AXContentTreeData>::SerializeChangedNodes(blink::WebAXObject, ui::AXTreeUpdateBase<content::AXContentNodeData, content::AXContentTreeData>*)
 
ui::AXTreeSerializer<blink::WebAXObject, content::AXContentNodeData, content::AXContentTreeData>::SerializeChangedNodes(blink::WebAXObject, ui::AXTreeUpdateBase<content::AXContentNodeData, content::AXContentTreeData>*)
 
ui::AXTreeSerializer<blink::WebAXObject, content::AXContentNodeData, content::AXContentTreeData>::SerializeChangedNodes(blink::WebAXObject, ui::AXTreeUpdateBase<content::AXContentNodeData, content::AXContentTreeData>*)
 
ui::AXTreeSerializer<blink::WebAXObject, content::AXContentNodeData, content::AXContentTreeData>::SerializeChangedNodes(blink::WebAXObject, ui::AXTreeUpdateBase<content::AXContentNodeData, content::AXContentTreeData>*)
 
ui::AXTreeSerializer<blink::WebAXObject, content::AXContentNodeData, content::AXContentTreeData>::SerializeChangedNodes(blink::WebAXObject, ui::AXTreeUpdateBase<content::AXContentNodeData, content::AXContentTreeData>*)
 
ui::AXTreeSerializer<blink::WebAXObject, content::AXContentNodeData, content::AXContentTreeData>::SerializeChangedNodes(blink::WebAXObject, ui::AXTreeUpdateBase<content::AXContentNodeData, content::AXContentTreeData>*)
 
ui::AXTreeSerializer<blink::WebAXObject, content::AXContentNodeData, content::AXContentTreeData>::SerializeChangedNodes(blink::WebAXObject, ui::AXTreeUpdateBase<content::AXContentNodeData, content::AXContentTreeData>*)
 
ui::AXTreeSerializer<blink::WebAXObject, content::AXContentNodeData, content::AXContentTreeData>::SerializeChangedNodes(blink::WebAXObject, ui::AXTreeUpdateBase<content::AXContentNodeData, content::AXContentTreeData>*)
 
ui::AXTreeSerializer<blink::WebAXObject, content::AXContentNodeData, content::AXContentTreeData>::SerializeChangedNodes(blink::WebAXObject, ui::AXTreeUpdateBase<content::AXContentNodeData, content::AXContentTreeData>*)
 
ui::AXTreeSerializer<blink::WebAXObject, content::AXContentNodeData, content::AXContentTreeData>::SerializeChangedNodes(blink::WebAXObject, ui::AXTreeUpdateBase<content::AXContentNodeData, content::AXContentTreeData>*)
 
ui::AXTreeSerializer<blink::WebAXObject, content::AXContentNodeData, content::AXContentTreeData>::SerializeChanges(blink::WebAXObject, ui::AXTreeUpdateBase<content::AXContentNodeData, content::AXContentTreeData>*)
 
content::RenderAccessibilityImpl::SendPendingAccessibilityEvents()
 
content::RenderAccessibilityImpl::OnEventsAck(int)
 
void IPC::DispatchToMethod<content::P2PSocketDispatcher, void (content::P2PSocketDispatcher::*)(int), void, std::__ndk1::tuple<int> >(content::P2PSocketDispatcher*, void (content::P2PSocketDispatcher::*)(int), void*, std::__ndk1::tuple<int>&&)
 
bool IPC::MessageT<AccessibilityMsg_Events_ACK_Meta, std::__ndk1::tuple<int>, void>::Dispatch<content::RenderAccessibilityImpl, content::RenderAccessibilityImpl, void, void (content::RenderAccessibilityImpl::*)(int)>(IPC::Message const*, content::RenderAccessibilityImpl*, content::RenderAccessibilityImpl*, void*, void (content::RenderAccessibilityImpl::*)(int))
 
content::RenderAccessibilityImpl::OnMessageReceived(IPC::Message const&)
 
content::RenderFrameImpl::OnMessageReceived(IPC::Message const&)
 
content::ChildThreadImpl::ChildThreadMessageRouter::RouteMessage(IPC::Message const&)
 
IPC::ChannelProxy::Context::OnDispatchMessage(IPC::Message const&)
 
base::debug::TaskAnnotator::RunTask(char const*, base::PendingTask*)
 
blink::scheduler::internal::ThreadControllerImpl::DoWork(blink::scheduler::internal::SequencedTaskSource::WorkType)
 
base::debug::TaskAnnotator::RunTask(char const*, base::PendingTask*)
 
base::internal::IncomingTaskQueue::RunTask(base::PendingTask*)
 
base::MessageLoop::RunTask(base::PendingTask*)
 
base::MessageLoop::DeferOrRunPendingTask(base::PendingTask)
 
base::MessageLoop::DoWork()
 
base::MessagePumpDefault::Run(base::MessagePump::Delegate*)
 
base::MessageLoop::Run(bool)
 
base::RunLoop::Run()
 
content::RendererMain(content::MainFunctionParams const&)
 
content::RunNamedProcessTypeMain(std::__ndk1::basic_string<char, std::__ndk1::char_traits<char>, std::__ndk1::allocator<char> > const&, content::MainFunctionParams const&, content::ContentMainDelegate*)
 
content::ContentMainRunnerImpl::Run()
 
service_manager::Main(service_manager::MainParams const&)



From what I can tell, this has nothing at all to do with the v8 file that causes the problem. Very confusing.
I'll fix this on the V8 side. Feel free to assign the bug to me if there are no other action items to be tracked here.
Forgot to update the bug: the llvm change was reverted, so for now we don't necessarily need to do anything about the miscompile. Might still be a good idea though.

Comment 56 by h...@chromium.org, Apr 13 2018

Regarding the stack trace: it seems likely that if V8 makes some JS code take a wrong turn somewhere, the crash could happen a little later, i.e. not with V8 on the stack, so it's not that surprising to me.

jkummerow: Thanks! I see you posted https://chromium-review.googlesource.com/c/v8/v8/+/1011651 I'll take a look at that now.
Project Member

Comment 57 by bugdroid1@chromium.org, Apr 17 2018

The following revision refers to this bug:
  https://chromium.googlesource.com/v8/v8.git/+/f7d6b19f6c674c056496b6995d2df5ef2684b9ae

commit f7d6b19f6c674c056496b6995d2df5ef2684b9ae
Author: Jakob Kummerow <jkummerow@chromium.org>
Date: Tue Apr 17 02:02:18 2018

[ubsan] Fix many static_cast<int32_t> with undefined behavior

Casting from a floating-point type to an integer type is undefined behavior
if the integral part of the float cannot be represented in the range of the
int.

Bug: v8:3770,  chromium:831145 
Cq-Include-Trybots: luci.chromium.try:linux_chromium_rel_ng;luci.v8.try:v8_linux_noi18n_rel_ng
Change-Id: I2e85ea8b0f09bbeeb3e0dcc1135fc747fa312f6d
Reviewed-on: https://chromium-review.googlesource.com/1011651
Commit-Queue: Jakob Kummerow <jkummerow@chromium.org>
Reviewed-by: Camillo Bruni <cbruni@chromium.org>
Cr-Commit-Position: refs/heads/master@{#52631}
[modify] https://crrev.com/f7d6b19f6c674c056496b6995d2df5ef2684b9ae/include/v8.h
[modify] https://crrev.com/f7d6b19f6c674c056496b6995d2df5ef2684b9ae/src/arm/simulator-arm.cc
[modify] https://crrev.com/f7d6b19f6c674c056496b6995d2df5ef2684b9ae/src/asmjs/asm-parser.cc
[modify] https://crrev.com/f7d6b19f6c674c056496b6995d2df5ef2684b9ae/src/base/ieee754.cc
[modify] https://crrev.com/f7d6b19f6c674c056496b6995d2df5ef2684b9ae/src/compiler/linkage.h
[modify] https://crrev.com/f7d6b19f6c674c056496b6995d2df5ef2684b9ae/src/compiler/machine-operator-reducer.cc
[modify] https://crrev.com/f7d6b19f6c674c056496b6995d2df5ef2684b9ae/src/conversions-inl.h
[modify] https://crrev.com/f7d6b19f6c674c056496b6995d2df5ef2684b9ae/src/conversions.h
[modify] https://crrev.com/f7d6b19f6c674c056496b6995d2df5ef2684b9ae/src/deoptimizer.cc
[modify] https://crrev.com/f7d6b19f6c674c056496b6995d2df5ef2684b9ae/src/ic/ic.cc
[modify] https://crrev.com/f7d6b19f6c674c056496b6995d2df5ef2684b9ae/src/mips/constants-mips.h
[modify] https://crrev.com/f7d6b19f6c674c056496b6995d2df5ef2684b9ae/src/mips64/constants-mips64.h
[modify] https://crrev.com/f7d6b19f6c674c056496b6995d2df5ef2684b9ae/src/objects.cc
[modify] https://crrev.com/f7d6b19f6c674c056496b6995d2df5ef2684b9ae/src/objects/intl-objects.cc
[modify] https://crrev.com/f7d6b19f6c674c056496b6995d2df5ef2684b9ae/src/runtime/runtime-numbers.cc
[modify] https://crrev.com/f7d6b19f6c674c056496b6995d2df5ef2684b9ae/src/runtime/runtime.h
[modify] https://crrev.com/f7d6b19f6c674c056496b6995d2df5ef2684b9ae/test/cctest/compiler/test-code-assembler.cc
[modify] https://crrev.com/f7d6b19f6c674c056496b6995d2df5ef2684b9ae/test/cctest/compiler/test-run-machops.cc
[modify] https://crrev.com/f7d6b19f6c674c056496b6995d2df5ef2684b9ae/test/cctest/test-api.cc
[modify] https://crrev.com/f7d6b19f6c674c056496b6995d2df5ef2684b9ae/test/cctest/test-code-stubs.cc
[modify] https://crrev.com/f7d6b19f6c674c056496b6995d2df5ef2684b9ae/test/cctest/wasm/test-run-wasm-simd.cc
[modify] https://crrev.com/f7d6b19f6c674c056496b6995d2df5ef2684b9ae/test/unittests/compiler/machine-operator-reducer-unittest.cc

Comment 58 by msw@chromium.org, Apr 17 2018

I'm not sure if this is the right bug to ping, but there are still a few flaky failures in telemetry_perf_unittests (memory_mobile and memory_desktop):

benchmarks.system_health_smoke_test.SystemHealthBenchmarkSmokeTest.system_health.memory_mobile.load:news:reddit
https://chromium-swarm.appspot.com/task?id=3cedcd700e982710&refresh=10&show_raw=1&wide_logs=true

benchmarks.system_health_smoke_test.SystemHealthBenchmarkSmokeTest.system_health.memory_mobile.load:tools:dropbox
https://chromium-swarm.appspot.com/task?id=3ce929c99b443d10&refresh=10&show_raw=1&wide_logs=true

benchmarks.system_health_smoke_test.SystemHealthBenchmarkSmokeTest.system_health.memory_desktop.browse:news:reddit
https://chromium-swarm.appspot.com/task?id=3ce8e20eca13b510&refresh=10&show_raw=1&wide_logs=true

Please let me know if there's another bug to cite/ping.

Comment 59 by h...@chromium.org, Apr 17 2018

Status: Fixed (was: Started)
msw: I see you also found crbug.com/833081, maybe that's a better place to track the flakiness.


This one turned out to not be a flake, but an actual bad interaction between V8 and a new Clang version.

Marking this fixed since we addressed the issue here.

It would be nice if the test was more user friendly (i.e. not requiring building in out/release specifically, providing less noise when hitting a DCHECK) but I'll leave that for other bugs.

Sign in to add a comment