New issue
Advanced search Search tips

Issue 902468 link

Starred by 1 user

Issue metadata

Status: Fixed
Owner:
Closed: Nov 27
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: Chrome
Pri: 2
Type: Bug



Sign in to add a comment

Chrome CQ doesn't perform debug builds of Chrome-OS-only code

Project Member Reported by derat@chromium.org, Nov 6

Issue description

https://crrev.com/c/1298417 passed on linux-chromeos-rel before it went in[1] but was reverted[2] due to a link error on linux-chromeos-dbg[3].

We appear to run various debug compile-only builders in the CQ (android_compile_dbg, linux_chromium_compile_dbg_ng, mac_chromium_compile_dbg_ng, win_chromium_compile_dbg_ng), but I don't see one for Chrome OS.

Dirk, should either linux_chromium_chromeos_compile_dbg_ng or linux-chromeos-compile-dbg be used to vet changes? I'm not sure about the difference between the two.

1. https://ci.chromium.org/p/chromium/builders/luci.chromium.try/linux-chromeos-rel/132097
2. https://crrev.com/c/1318546
3. https://ci.chromium.org/p/chromium/builders/luci.chromium.ci/linux-chromeos-dbg/8728
 
Cc: dpranke@chromium.org
Owner: ----
Status: Available (was: Untriaged)
Yes, it probably makes sense to have one of them; I'm not sure if it matters much which. 

It would be interesting to see if we had any idea how often we get debug-only compile failures. If it's only once every couple months, it probably wouldn't be worth it to add it to the CQ.
Well, I would suggest that we should make that call equally across all platforms and not single out CrOS :)

Looking at the last 200 linux-chromeos-dbg builds (about 10 days), I see 5 "new" cases of "Failed compile" (i.e. where the previous run didn't also fail) which is pretty high, 2-3 times / week.




Cc: jbudorick@chromium.org
Owner: bpastene@chromium.org
Status: Assigned (was: Available)
And it looks like everytime compile went red on the dbg bot, it took a revert of a previously landed CL to get it green. Given its frequency, I think this is a good case for CQ promotion.

I'll add linux-chromeos-compile-dbg to the CQ unless dirk/jbud disagree. (Not sure where linux_chromium_chromeos_compile_dbg_ng comes from... might be an old bot I can throw away.)
Also.. given how many bots I've added to the CQ recently, and given how close we are to our GCE quotas in some zones, I'm going to use this as an opportunity to trim up some bot pools so we can use the freed capacity for linux-chromeos-compile-dbg's transition to CQ.
Project Member

Comment 5 by bugdroid1@chromium.org, Nov 7

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/3b9a30d2b89c0859b5556281c73e84816f818ab3

commit 3b9a30d2b89c0859b5556281c73e84816f818ab3
Author: Ben Pastene <bpastene@chromium.org>
Date: Wed Nov 07 01:15:16 2018

Remove old linux-chromeos bots from buildbucket and milo.

All of these were either renamed, or removed a while ago.

R=martiniss, jbudorick

Bug:  902468 
Change-Id: I8f880c8b63fbbffca985eeef2ed273a75dff5d2a
Reviewed-on: https://chromium-review.googlesource.com/c/1321732
Reviewed-by: John Budorick <jbudorick@chromium.org>
Commit-Queue: Ben Pastene <bpastene@chromium.org>
Cr-Commit-Position: refs/heads/master@{#605913}
[modify] https://crrev.com/3b9a30d2b89c0859b5556281c73e84816f818ab3/infra/config/global/cr-buildbucket.cfg
[modify] https://crrev.com/3b9a30d2b89c0859b5556281c73e84816f818ab3/infra/config/global/luci-milo.cfg

re #c2:
> Well, I would suggest that we should make that call equally across all platforms
> and not single out CrOS :)

We don't currently have debug compile builders for Cast, iOS, or Fuchsia either, 
so CrOS isn't being singled out. The questions I'm asking are the questions we 
should ask for any new CQ config.

That said, 2-3 times a week is enough to justify it.
Yeah, it's surprisingly easy to break only the debug builds, even without debug-specific code. In the recent example from comment 1, my guess is that the linker error never happened in release builds because the relevant constants were inlined.
Project Member

Comment 8 by bugdroid1@chromium.org, Nov 16

The following revision refers to this bug:
  https://chrome-internal.googlesource.com/infradata/config/+/c01a4256238f4a37a408b9311cb94873f8b819e5

commit c01a4256238f4a37a408b9311cb94873f8b819e5
Author: Ben Pastene <bpastene@chromium.org>
Date: Fri Nov 16 23:15:32 2018

Project Member

Comment 9 by bugdroid1@chromium.org, Nov 16

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/1d72be2f8c97328baf61e6c04819c75508b22829

commit 1d72be2f8c97328baf61e6c04819c75508b22829
Author: Ben Pastene <bpastene@chromium.org>
Date: Fri Nov 16 23:36:26 2018

Add linux-chromeos-compile-dbg to the CQ at 50% experiment.

Hopefully should have enough bots:
https://luci-milo.appspot.com/p/chromium/builders/luci.chromium.try/linux-chromeos-compile-dbg

R=jbudorick, dpranke

Bug:  902468 
Change-Id: I9273d96ede4c056d8bd4b936f5f843176bac087a
Reviewed-on: https://chromium-review.googlesource.com/c/1340536
Reviewed-by: Dirk Pranke <dpranke@chromium.org>
Reviewed-by: John Budorick <jbudorick@chromium.org>
Commit-Queue: Ben Pastene <bpastene@chromium.org>
Cr-Commit-Position: refs/heads/master@{#609025}
[modify] https://crrev.com/1d72be2f8c97328baf61e6c04819c75508b22829/infra/config/branch/cq.cfg

Project Member

Comment 10 by bugdroid1@chromium.org, Nov 20

The following revision refers to this bug:
  https://chrome-internal.googlesource.com/infradata/config/+/545dc1f41a5289bd43a8f7ede4190644701ad0eb

commit 545dc1f41a5289bd43a8f7ede4190644701ad0eb
Author: Ben Pastene <bpastene@chromium.org>
Date: Tue Nov 20 00:03:43 2018

Project Member

Comment 11 by bugdroid1@chromium.org, Nov 20

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/f366637606f29058ad756abb812d0bf75f2ca61a

commit f366637606f29058ad756abb812d0bf75f2ca61a
Author: Ben Pastene <bpastene@chromium.org>
Date: Tue Nov 20 23:05:56 2018

Bump linux-chromeos-compile-dbg CQ bot to 100% experimental.

Should have enough bots.

R=jbudorick, dpranke

Bug:  902468 
Change-Id: If699354b2747c77a11741972fe8b531d9d500234
Reviewed-on: https://chromium-review.googlesource.com/c/1344689
Reviewed-by: Dirk Pranke <dpranke@chromium.org>
Commit-Queue: Ben Pastene <bpastene@chromium.org>
Cr-Commit-Position: refs/heads/master@{#609841}
[modify] https://crrev.com/f366637606f29058ad756abb812d0bf75f2ca61a/docs/infra/cq_builders.md
[modify] https://crrev.com/f366637606f29058ad756abb812d0bf75f2ca61a/infra/config/branch/cq.cfg

Project Member

Comment 12 by bugdroid1@chromium.org, Nov 27

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/42528e7962c528403035c5f78be9e0a0710a5afd

commit 42528e7962c528403035c5f78be9e0a0710a5afd
Author: Ben Pastene <bpastene@chromium.org>
Date: Tue Nov 27 22:22:17 2018

Promote linux-chromeos-compile-dbg to fully-blocking CQ bot.

R=dpranke, jbudorick

Bug:  902468 
Change-Id: Id7e4909528758fd24a22e00696e905346a06b950
Reviewed-on: https://chromium-review.googlesource.com/c/1352267
Reviewed-by: Dirk Pranke <dpranke@chromium.org>
Commit-Queue: Ben Pastene <bpastene@chromium.org>
Cr-Commit-Position: refs/heads/master@{#611371}
[modify] https://crrev.com/42528e7962c528403035c5f78be9e0a0710a5afd/docs/infra/cq_builders.md
[modify] https://crrev.com/42528e7962c528403035c5f78be9e0a0710a5afd/infra/config/branch/cq.cfg

Status: Fixed (was: Assigned)
After #12, https://luci-milo.appspot.com/p/chromium/builders/luci.chromium.try/linux-chromeos-compile-dbg should now be fully on chrome's CQ. (And sourced entirely via recycled VMs. Who says we aren't sustainable ;)

LMK if there's anything else, thanks.

Sign in to add a comment