Support perf bisects for broken builds |
|||||
Issue descriptionScenario: The build is broken for 200 commits, then once it's fixed we get a binary size alert. Normally we use tools/binary_size/diagnose_bloat.py to pinpoint any regressions but this takes forever locally. Does perf bisecting support anything like this? Would look like: 1. Recognize that the build is broken 2. Once the build is fixed, performs a bisect for the broken range with the fixing CL applied, and generates new alerts from this I'm assuming this probably isn't supported or even feasible, but just wanted to check! Issue 839531 is an example of this where the range is about 170 commits.
,
May 3 2018
Wow, that's really cool! Re swarming: What are the requirements for this? Any docs I could take a look at?
,
May 3 2018
+eyaich I don't know all the specifics of how to get things running in swarming, there's an entry point script that Emily maintains that runs the perf test in question, and the builder builds the isolate w/ all the files necessary for your perf test. Dave and Emily can probably help get you started if this is something you're interested in.
,
May 3 2018
Just writing +eyaich doesn't make it happen.
,
May 3 2018
Oh, yes, all of this sounds doable! There are a few parts: * Running a bisect with a patch: Landed this week! But the UI isn't ready yet. I think Simon is talking about kicking one off with a dev console. * Running a perf bisect on a build without running tests: Should be generally supported, but there's some minor tweaks we need to make. * Running resource_sizes. This is a little tricky, since they're run on "Android Builder Perf", but Pinpoint only has "Android Compile Perf" bots. The person who was going to migrate them over left the team before they completed that work. (Issue 807273)
,
May 3 2018
I don't think we would involve Swarming at all.
,
May 3 2018
Ah the builder runs the "perf tests" in this case? Then yeah, don't even need swarming, simpler :)
,
Jun 13 2018
simonhatch@ and dtu@, has anything advanced on this front that isn't reflected on the bug? The reason for asking is the same question as before - a binary size regression somewhere in a range of 1137 commits. Is the block issue (waiting on "Android Compile" to be converted to LUCI) still actually blocking?
,
Jun 14 2018
Can resource_sizes be run in Swarming? It might be easier to integrate if we can run it that way, rather than coming up with some scheme to pull and store the results from the builders.
,
Jun 14 2018
supersize requires most of the out/Release directory (all .o files, most .jar files, lib.unstripped/libchrome.so, all .ninja files). It also now requires all OWNERS files as well. While we could upload all of this to swarming, it's probably better that we don't. We could alternatively run it locally and then run a "swarming test" that has the result as its inputs, and just cp's it to the expected result.json location. Note that this would have a side-benefit of more efficiently archiving the .size files (as swarming inputs rather than as as part of the 1GB build output .zip). We'd still want this executing from the builder though, so that we get per-commit resolution. Do you know of any reason that there would be hiccups to add a swarming test to the builder?
,
Yesterday
(45 hours ago)
|
|||||
►
Sign in to add a comment |
|||||
Comment 1 by simonhatch@chromium.org
, May 3 2018