Recipe links for a swarmed test are misleading |
||||||
Issue descriptionProblem: When a user looks at a test shard failure, there's: - too many links - the test 'stdout' log is misleading Use case: Taking a simple failure: https://ci.chromium.org/p/chromium/builders/luci.chromium.try/win7_chromium_rel_ng/46920 base_unittests runs a single shard, with a single test case (ScopedTempDir.MatchesTempEnvVar) that failed. Yet it has - stdout - swarming.summary - Merge_script_log - ScopedTempDir.MatchesTempEnvVar - step_metadata - shard #0 isolated_out - shard #0 (failed) That's too much. - 'stdout' is actively hurting. It presents the recipe environment variable at the top, then dump the log of the shard. Users are confused by this, they think the environment variables were the ones used when the task ran, but this isn't true. - 'swarming.summary' doesn't have value. The Swarming task web UI is better suited. - 'Merge script log' is useless. - 'ScopedTempDir.MatchesTempEnvVar' is useful. - 'step_metadata' is useless IMHO. It's confusing for users. - 'shard #0 isolated out' is not needed, users can find this information through the Swarming task web UI. - 'shard #0 (failed)' is useful. AI: - Remove 'swarming.summary', 'Merge script log', 'step_metadata', 'shard #0 isolated out'. - Find a way to not print the environment variables in 'stdout'. That would be a one-off.
,
Jul 25
The "Merge_script_log" was instrumental in debugging the test step I/O timed out (see https://bugs.chromium.org/p/chromium/issues/detail?id=856712#c26)
,
Jul 25
Most of the time, Merge_script_log will not be useful. However for test with complex merge logic, it's very important. I am happy to make a CL changing the recipe code to not show Merge_script_log if that's mostly empty/whitespace-ish. What do folks think?
,
Jul 25
I would also chime in 'shard #0 isolated out' is useful for case the swarming logs are too huge & people would just want to quickly navigate to the output directory. If swarming add something like a pausing log loading, then I would agree with removing 'shard #0 isolated out' +crouleau@ who rely on 'shard #0 isolated out' recently
,
Jul 25
I think the biggest priority here is removing the environment variables. Extra detail is easy to scroll past, but the env vars are actually incorrect.
,
Jul 25
As someone who has been using this interface infrequently over the past 2.5 years as it has evolved, I have to say that I never know which links to click on. I also frequently hit broken links. So if there is some way to highlight the links that users should (1) expect to work all the time and not be broken and (2) use more frequently than other links, then that would be very helpful. Extra detail is painful to filter out. Also, the swarming dashboard (this one: https://chromium-swarm.appspot.com/task?id=3eead4276c439a10&refresh=10&show_raw=1 ) fails to load if there is too much output for Chrome to handle. I never would have guessed that I should click the "isolated out" link instead of the "shard #0" link to get a functional dashboard.
,
Jul 25
Re #6, the task you linked so has a very short output, so I think you mistakenly copied the wrong task URL. I filed issue 867621 for this problem, which is orthogonal and can be fixed independently. Re #3 'Merge script log', I think it's a great idea to only show it when there's actual content. Re #5, that's true.
,
Aug 1
,
Oct 18
,
Oct 18
|
||||||
►
Sign in to add a comment |
||||||
Comment 1 by martiniss@chromium.org
, Jul 25