New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 879389 link

Starred by 1 user

Issue metadata

Status: Assigned
Owner:
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: ----
Pri: 2
Type: Bug



Sign in to add a comment

Code coverage: extend the test suite

Project Member Reported by mmoroz@chromium.org, Aug 30

Issue description

It looks like there might be some other tests that we're missing in https://cs.chromium.org/chromium/src/tools/code_coverage/test_suite.txt. It would be nice to add those.
 
Project Member

Comment 1 by bugdroid1@chromium.org, Aug 30

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/001bd26b5f7782052c5b9c4e4508714897b1eebc

commit 001bd26b5f7782052c5b9c4e4508714897b1eebc
Author: Max Moroz <mmoroz@chromium.org>
Date: Thu Aug 30 23:25:04 2018

[Code Coverage] Add zucchini_unittests to the test suite.

TBR=inferno@chromium.org,liaoyuke@chromium.org

Bug: 879389
Change-Id: I74ab663e1b7e69457d6f81cfa6890422610f4c9a
Reviewed-on: https://chromium-review.googlesource.com/1197829
Commit-Queue: Max Moroz <mmoroz@chromium.org>
Reviewed-by: Yuke Liao <liaoyuke@chromium.org>
Reviewed-by: Max Moroz <mmoroz@chromium.org>
Cr-Commit-Position: refs/heads/master@{#587837}
[modify] https://crrev.com/001bd26b5f7782052c5b9c4e4508714897b1eebc/tools/code_coverage/test_suite.txt

Cc: dpranke@chromium.org jbudorick@chromium.org aluo@chromium.org yihongg@chromium.org
CC'ing more people who might have ideas where we can find more tests. Please share if you think we're missing anything in: https://cs.chromium.org/chromium/src/tools/code_coverage/test_suite.txt

Also, does anyone think we should try running everything produced by:

"gn refs out/coverage //base/test:test_support -t executable"

?
maintaining a separate text file seems like not the best way to manage this.

Ideally either you'd get something like your `gn refs` call to work, or define a single "//:all_tests" target and maintain that list instead.
Cc: st...@chromium.org
Thanks, Dirk! I've also talked to jbudorick@ today and we agreed that some of those targets might be broken / useless / etc. It doesn't look like any of the existing testbots is running all those things, but it might be interesting to run all of them. At least, without code coverage first, just to see how bad or good thing are.
Wondering whether it's feasible now to align this with the CQ trybot test spec.
That's the direction I'd like to move to with the redesign to using Luci infra.
stgao@, that sounds good too. Could you please share (if you know) which tests the existing CQ bots are running? I think the existing test_suite was based on some of the bots, but I wonder if things have changed or we are missing anything.

It still a good question though, why do we have ~400 test targets if we run only ~100 of them? Should we run more or delete some...

I don't know the details, but the test spec are in https://cs.chromium.org/chromium/src/testing/buildbot/
It might not doable if the current setup is not on Recipes.
Blocking: -759794
Owner: st...@chromium.org
Status: Assigned (was: Available)
Handing this over to Shuotao, as per c#7-c#8. It would be awesome to extend the test suite by using more or less unified set of tests instead of the existing https://cs.chromium.org/chromium/src/tools/code_coverage/test_suite.txt that was compiled mostly manually based on some testbot and inquiries from developers.

Sign in to add a comment