New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 851146 link

Starred by 4 users

Issue metadata

Status: Available
Owner: ----
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: Linux , Android
Pri: 2
Type: Bug



Sign in to add a comment

"package build" step slowly increasing in duration for some builders; now at ~2 hours

Project Member Reported by bpastene@chromium.org, Jun 8 2018

Issue description

See https://luci-milo.appspot.com/p/chromium/builders/luci.chromium.ci/Linux%20Builder?limit=1000 where cycle time went from ~20min a few weeks ago to now about 2.5 hours. All due to the package build step.

Something's probably not getting cleared out from the build dir. Will look into it.
 
chrome-bot@swarm965-c4:(Linux 14.04):/b/swarming$ ls -alh /b/swarming/w/ir/cache/builder/src/out/Release/chromium-browser-beta*
-rw-r--r-- 1 chrome-bot chrome-bot 186M Apr  8 08:49 /b/swarming/w/ir/cache/builder/src/out/Release/chromium-browser-beta_67.0.3391.0-1_amd64.deb
-rw-r--r-- 1 chrome-bot chrome-bot 186M Apr  8 08:49 /b/swarming/w/ir/cache/builder/src/out/Release/chromium-browser-beta-67.0.3391.0-1.x86_64.rpm
-rw-r--r-- 1 chrome-bot chrome-bot 186M Apr  8 20:10 /b/swarming/w/ir/cache/builder/src/out/Release/chromium-browser-beta_67.0.3392.0-1_amd64.deb
-rw-r--r-- 1 chrome-bot chrome-bot 186M Apr  8 20:10 /b/swarming/w/ir/cache/builder/src/out/Release/chromium-browser-beta-67.0.3392.0-1.x86_64.rpm
-rw-r--r-- 1 chrome-bot chrome-bot 187M Apr  9 20:34 /b/swarming/w/ir/cache/builder/src/out/Release/chromium-browser-beta_67.0.3393.0-1_amd64.deb
-rw-r--r-- 1 chrome-bot chrome-bot 187M Apr  9 20:34 /b/swarming/w/ir/cache/builder/src/out/Release/chromium-browser-beta-67.0.3393.0-1.x86_64.rpm
-rw-r--r-- 1 chrome-bot chrome-bot 187M Apr 10 20:07 /b/swarming/w/ir/cache/builder/src/out/Release/chromium-browser-beta_67.0.3394.0-1_amd64.deb
-rw-r--r-- 1 chrome-bot chrome-bot 187M Apr 10 20:07 /b/swarming/w/ir/cache/builder/src/out/Release/chromium-browser-beta-67.0.3394.0-1.x86_64.rpm
-rw-r--r-- 1 chrome-bot chrome-bot 187M Apr 11 20:01 /b/swarming/w/ir/cache/builder/src/out/Release/chromium-browser-beta_67.0.3395.0-1_amd64.deb
-rw-r--r-- 1 chrome-bot chrome-bot 187M Apr 11 20:01 /b/swarming/w/ir/cache/builder/src/out/Release/chromium-browser-beta-67.0.3395.0-1.x86_64.rpm
-rw-r--r-- 1 chrome-bot chrome-bot 187M Apr 12 18:59 /b/swarming/w/ir/cache/builder/src/out/Release/chromium-browser-beta_67.0.3396.0-1_amd64.deb
-rw-r--r-- 1 chrome-bot chrome-bot 187M Apr 12 18:59 /b/swarming/w/ir/cache/builder/src/out/Release/chromium-browser-beta-67.0.3396.0-1.x86_64.rpm
-rw-r--r-- 1 chrome-bot chrome-bot 187M Apr 13 20:08 /b/swarming/w/ir/cache/builder/src/out/Release/chromium-browser-beta_68.0.3397.0-1_amd64.deb
-rw-r--r-- 1 chrome-bot chrome-bot 187M Apr 13 20:08 /b/swarming/w/ir/cache/builder/src/out/Release/chromium-browser-beta-68.0.3397.0-1.x86_64.rpm
-rw-r--r-- 1 chrome-bot chrome-bot 186M Apr 16 20:03 /b/swarming/w/ir/cache/builder/src/out/Release/chromium-browser-beta_68.0.3398.0-1_amd64.deb
-rw-r--r-- 1 chrome-bot chrome-bot 186M Apr 16 20:03 /b/swarming/w/ir/cache/builder/src/out/Release/chromium-browser-beta-68.0.3398.0-1.x86_64.rpm
-rw-r--r-- 1 chrome-bot chrome-bot 185M Apr 17 20:00 /b/swarming/w/ir/cache/builder/src/out/Release/chromium-browser-beta_68.0.3399.0-1_amd64.deb
-rw-r--r-- 1 chrome-bot chrome-bot 185M Apr 17 19:59 /b/swarming/w/ir/cache/builder/src/out/Release/chromium-browser-beta-68.0.3399.0-1.x86_64.rpm
-rw-r--r-- 1 chrome-bot chrome-bot 187M Apr 18 20:10 /b/swarming/w/ir/cache/builder/src/out/Release/chromium-browser-beta_68.0.3400.0-1_amd64.deb
-rw-r--r-- 1 chrome-bot chrome-bot 187M Apr 18 20:10 /b/swarming/w/ir/cache/builder/src/out/Release/chromium-browser-beta-68.0.3400.0-1.x86_64.rpm

Looks like a bunch of browser package/installation files are accumulating over time. Each about 200M. Certainly not helping.
All those files total about 71G. Not surprisingly, a new set gets created every time chrome/VERSION changes:
https://chromium.googlesource.com/chromium/src/+log/master/chrome/VERSION
Cc: thestig@chromium.org thomasanderson@chromium.org
Cc: gov...@chromium.org
Labels: OS-Linux
+govind
Currently the Linux package files have the version in the filename, but this means they don't get cleaned up when the version increments, leading to huge build dirs like on the Linux Builder.

If we change the filename to something deterministic like chromium-browser-stable.deb, would the release team be able to handle this change easily?
I rm'ed all chromium-browser-* files in the bot's checkout and it's back down to 20min cycle times:
https://luci-milo.appspot.com/p/chromium/builders/luci.chromium.ci/Linux%20Builder/102817

So the slow-down is certainly due to those files.
If the package_build step was smarter about what .deb / .rpm files to package, that can avoid the slowdown, but not the disk space usage.
Cc: abdulsyed@chromium.org mmoss@chromium.org
+ abdulsyed@ & mmoss@, PTAL comment #4. Thank you.
The long term right answer is to get rid of the step package_build. We should only send isolated hashes around.

Comment 9 by mmoss@chromium.org, Jun 11 2018

Re #4, no, release could not easily handle a change in the package names, and I don't think that's the right fix anyhow. Maybe the build system should have an option (if it doesn't already) to clean up files that it doesn't know about? Accumulated build cruft isn't exclusively a problem of package files (e.g. from file renames, obsolete components, etc.). The packages just highlight the issue because of the frequency of name changes and their size.

And on a related note, why are those packages so large? 200MB is like 4X the size of official packages. Are the binaries not being stripped or something?
IMHO, I still stand by my c#4 that the package output name should be deterministic, but it seems folks disagree.  +dpranke does the build system allow us to do what mmoss is suggesting in c#9?
Cc: friedman@chromium.org
The build system does not currently have that option, no.

We should get rid of package_build in favor of isolated hashes as maruel@ says, but that wouldn't solve the underlying problem of unbounded disk growth on the builder.

I thought the disk cleanup tool should, though. +friedman, am I misremembering that?
Here's the list of all paths cleanup_disk watches:
https://chrome-internal.googlesource.com/infra/infra_internal/+/master/go/src/infra_internal/tools/cleanup_disk/cmd/cleanup_disk/main.go#102

I don't think we've ever had that touching build dirs (maybe it used to be covered by the "build.dead" dir that buildbot generates?). We have a step called "clean local files" running in the early-parts of every android-flavored chromium build (eg: https://ci.chromium.org/p/chromium/builders/luci.chromium.ci/android-kitkat-arm-rel/6616)
https://codesearch.chromium.org/chromium/build/scripts/slave/recipe_modules/chromium_android/api.py?rcl=1aaf79598cf2a5b44e98e75003b95ed31f8f09da&l=151

It's old and likely obsolete, but we could generalize that for every chromium build, and tweak it to target these package files. If dpranke/jbud endorse that, I can work on adding it.
buildbot used build.dead to manage directories for builders that no longer existed on a bot.

One relatively easy answer would be to say that if we're building an installer, the build should be a clobber build, but that might be overly draconian.

I'd rather not have an ad-hoc thing that just deleted a few files, so the only other option I see is to do something based on the build graph itself, sort of like an inverted `ninja -t clean` (delete everything *except* the stuff in the build graph). 

That functionality doesn't currently exist but probably wouldn't be too hard to add to either ninja or GN (both of which could do it).


Labels: OS-Android
Any updates on this? I suspect this problem is what's killing chromium-internal-arm-builder-rel

https://ci.chromium.org/buildbot/internal.client.clank/chromium-internal-arm-builder-rel/?limit=200
Not entirely sure it's related. The duration of package_build seems pretty consistent, but it gets stuck at some point and hangs with no stdout for 20min then gets killed. Whatever it is, I clobbered the bot. So that'll prob help.

As for this bug, it can mostly be duped into bug 754104 at this point. Though I think I recall dpranke following up on his build graph idea in some other bug...
Labels: -Pri-1 Pri-2
Owner: ----
Status: Available (was: Assigned)
FTR, just ran into the same thing in  bug 880932 . Clobbered the bot https://ci.chromium.org/p/chromium/builders/luci.chromium.ci/Linux%20Builder and its package_build step duration went from 2hr to 7min.

Releasing this since I'm not actively tracking down & fixing what's generating them. Also lowering pri since this becomes much less disruptive once we've killed the package_build step (bug 754104)
Cc: drott@chromium.org
 Issue 918123  has been merged into this issue.
I ran into this over the holiday break. The .deb and .rpm files built up on the bot since the end of October, and started breaking the bot.

Not sure what to do here...
Cc: martiniss@chromium.org

Sign in to add a comment