Friendly ping?

Will take a look.

This tool fetch Chrome snapshot from snapshot archiver, like
http://commondatastorage.googleapis.com/chromium-browser-snapshots/Linux_ChromiumOS_Full/427025/chrome-linux.zip

However, the snapshot is not universal for every CrOS boards (X64 binary). And the tool doesn't have ability to deploy the snapshot to a CrOS device (like deploy_chrome in chrome-sdk shell http://www.chromium.org/chromium-os/how-tos-and-troubleshooting/building-chromium-browser#TOC-Using-deploy_chrome). I'm checking if there's a trivial way to deploy X64 binary to X86 boards.

I also found some issues are related to this:
https://bugs.chromium.org/p/chromium/issues/detail?id=339193
https://bugs.chromium.org/p/chromium/issues/detail?id=339188

Dean, afaiu what's pulled from the archive is a Chrome OS-on-Linux build, i.e. compiled with target_os="chromeos".  It should run on Linux perfectly fine, no need to deploy to a device.

Thiemo, thanks. I tried the command you provided on Linux workstation. For the first one (revision 424009) it crashed (SIGSIGV). However, I accidentally said it is a good one, and the next try (revision 430623) can render properly.

To avoid such silently behaviour, I'm working on a CL to emit more debug message. And for returncode < 0 (signal received), the revision result should be marked as bad automatically. 

Thank you Dean, that's super helpful.

Digging a bit into this, here's where we recover from the breakage (I'd guess it's the ICU CL):

You are probably looking for a change made after 429311 (known bad), but no later than 429321 (first known good).
CHANGELOG URL: https://chromium.googlesource.com/chromium/src/+log/25728d1bd2994ad9da88d3139e82917c2aa23198..e2c5d8b5d9faba78784799370c1542c165f14ac9

Going further into the past, 410623, 400623, 350655, 300627, 252459 all are crashing.  Imho there would be great value in fixing this.

I hope that it's just crashing because a path to some kind of resource that has changed.  Looking at my notes, in the past I could fix some crashes by:
$ mkdir -p third_party/icu/source/data/in
$ ln -s third_party/icu/common/icudtl.dat third_party/icu/source/data/in/icudtl.dat

Case in point: After checking out revision 429311, the situation is inverted.  The formerly crashing versions are working whereas the formerly working versions are crashing:

$ git checkout 25728d1bd2994ad9da88d3139e82917c2aa23198
$ gclient sync -j100
$ python tools/bisect-builds.py -a chromeos --verify-range -g 429311
# WORKS
$ python tools/bisect-builds.py -a chromeos --verify-range -g 429321
# FAILS

Thus it really seems that icudtl.dat has been moved and all that's required to fix the crashes is to put the file in the correct location, depending on the Chrome revision.

+1 for this. This would definitely help the test team with bisects of critical issues.
Currently manual binary bisects take about 10 mins per iteration(ie, time taken to download the image + image the test device.) 

This came up again for File Manager, see issue 826131.

See also  issue 789612 ,  issue 764487 ,  issue 666319 ,  issue 600740 . Fixing this tool (and maybe adding tests so it doesn't regress) would help a lot of people.

 Issue 666319  has been merged into this issue.

 Issue 884422  has been merged into this issue.

 Issue 600740  has been merged into this issue.

The initial report was fixed for tools/bisect-builds.py in r520210

upstreaming those fixes to chrome/test/python_tests/bisect_builds.py results in a new error (common to both scripts), since Jan, which causes every Renderer to crash. It goes

#
# Fatal error in , line 0
# Check failed: (isolate->snapshot_blob()) == nullptr.
#
#
#
#FailureMessage Object: 0x7ffe81d100b0#0 0x558d14a13b6c base::debug::StackTrace::StackTrace()
#1 0x558d171fa67b gin::(anonymous namespace)::PrintStackTrace()
#2 0x558d171f36f8 V8_Fatal()
#3 0x558d13ebd3fa v8::IsolateNewImpl()
#4 0x558d171f7082 gin::IsolateHolder::IsolateHolder()
#5 0x558d17128743 blink::V8PerIsolateData::V8PerIsolateData()
#6 0x558d171292a8 

Bisecting on that suggests this regressed in r531204 "chrome: Unship snapshot_blob.bin from Win/Mac/Linux"

peria@: could take a look?

Unfortunately, these files are versioned, so there's no easy way to just copy it from a checkout the way we do for a couple of other things.

I have a change in https://chrome-internal-review.googlesource.com/c/chrome/test/python_tests/+/682748 which upstreams some fixes to python_tests, but i'm not sure what to do about the v8 failure.

A small tweak to r531204 may fix future builds.

Ideally we can recover the range from ~January when this started failing too :/. (not having the ability to bisect really hurts productivity!). Maybe you have some ideas.

Seems this latest problem is similar to Issue 815049

Theory: maybe we can download both the chromeos build and some other platform and use its snapshot_blob.bin ..

Update: theory checks out. In case anybody else gets desperate, I've successfully done a bisect with some mad hacks at

https://chrome-internal-review.googlesource.com/c/chrome/test/python_tests/+/682748

If it can't find snapshot_blob.bin after extracting the chromeos build, it will:
 - try to find a "nearby" desktop-linux revision
 - download it, and copy its v8_context_snapshot.bin to snapshot_blob.bin
 - hope the v8 version matches..


That CL also adds a "--verbose" option. E.g.

 [chromium/src]: python chrome/test/python_tests/bisect_builds.py --verbose --use-local-cache -a chromeos --good=550428 --bad=561733

IIUC, bundled files for internal bisection are managed in infra-team, and if they don't exist for some specific builds, someone in infra team needs to upload them.
prasadv@, could you take a look?  I think this issue is based on the same reason for issue 815049.


Beside it, it looks strange that CrOS does not have snapshot_blob.bin, because we don't build not use v8_snapshot_context.bin on CrOS. Does it refer Linux's settings?