New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 839543 link

Starred by 2 users

Issue metadata

Status: Available
Owner: ----
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: Android
Pri: 3
Type: Bug

Blocked on:
issue 807273



Sign in to add a comment

Support perf bisects for broken builds

Project Member Reported by estevenson@chromium.org, May 3 2018

Issue description

Scenario:

The build is broken for 200 commits, then once it's fixed we get a binary size alert. Normally we use tools/binary_size/diagnose_bloat.py to pinpoint any regressions but this takes forever locally.

Does perf bisecting support anything like this? Would look like:
1. Recognize that the build is broken
2. Once the build is fixed, performs a bisect for the broken range with the fixing CL applied, and generates new alerts from this

I'm assuming this probably isn't supported or even feasible, but just wanted to check!

 Issue 839531  is an example of this where the range is about 170 commits.
 
Cc: dtu@chromium.org
+dtu

Yes! This is almost ready to go :) In fact, Dave landed initial support last week, might be able to try manually starting a job with the fix on 839531 although I'm not clear if resource sizes is something we can run via swarming.

https://github.com/catapult-project/catapult/issues/4335

Wow, that's really cool!

Re swarming: What are the requirements for this? Any docs I could take a look at?
+eyaich

I don't know all the specifics of how to get things running in swarming, there's an entry point script that Emily maintains that runs the perf test in question, and the builder builds the isolate w/ all the files necessary for your perf test. Dave and Emily can probably help get you started if this is something you're interested in.
Cc: eyaich@chromium.org
Just writing +eyaich doesn't make it happen.

Comment 5 by dtu@chromium.org, May 3 2018

Blockedon: 807273
Oh, yes, all of this sounds doable! There are a few parts:

* Running a bisect with a patch: Landed this week! But the UI isn't ready yet. I think Simon is talking about kicking one off with a dev console.

* Running a perf bisect on a build without running tests: Should be generally supported, but there's some minor tweaks we need to make.

* Running resource_sizes. This is a little tricky, since they're run on "Android Builder Perf", but Pinpoint only has "Android Compile Perf" bots. The person who was going to migrate them over left the team before they completed that work. (Issue 807273)

Comment 6 by dtu@chromium.org, May 3 2018

I don't think we would involve Swarming at all.
Ah the builder runs the "perf tests" in this case? Then yeah, don't even need swarming, simpler :)
simonhatch@ and dtu@, has anything advanced on this front that isn't reflected on the bug?

The reason for asking is the same question as before - a binary size regression somewhere in a range of 1137 commits.

Is the block issue (waiting on "Android Compile" to be converted to LUCI) still actually blocking?

Comment 9 by dtu@chromium.org, Jun 14 2018

Can resource_sizes be run in Swarming? It might be easier to integrate if we can run it that way, rather than coming up with some scheme to pull and store the results from the builders.
Status: Available (was: Untriaged)
supersize requires most of the out/Release directory (all .o files, most .jar files, lib.unstripped/libchrome.so, all .ninja files). It also now requires all OWNERS files as well. While we could upload all of this to swarming, it's probably better that we don't.

We could alternatively run it locally and then run a "swarming test" that has the result as its inputs, and just cp's it to the expected result.json location. Note that this would have a side-benefit of more efficiently archiving the .size files (as swarming inputs rather than as as part of the 1GB build output .zip).

We'd still want this executing from the builder though, so that we get per-commit resolution. Do you know of any reason that there would be hiccups to add a swarming test to the builder?

Comment 11 by agrieve@chromium.org, Yesterday (45 hours ago)

Labels: -Performance-Size

Sign in to add a comment