Perf swarmed logs are really hard to find from perf dashboard |
|||||||||||
Issue descriptionHere's a chart on perf dashboard from a swarmed bot: https://chromeperf.appspot.com/group_report?bug_id=663141 If you click on a data point, then click the tooltip, then click "Buildbot stdio", the url is something like this: https://build.chromium.org/p/chromium.perf/builders/Mac%20Pro%2010.11%20Perf/builds/76/steps/page_cycler_v2.intl_es_fr_pt-BR/logs/stdio That's incorrect. The buildbot status page is at https://build.chromium.org/p/chromium.perf/builders/Mac%20Pro%2010.11%20Perf/builds/76 And then it's hard to figure out how to get to the logs from there: 1) You want stdout, not stdio (which is what we're used to clicking on) 2) Sometimes when I click I get a page like this: https://chromium-swarm.appspot.com/user/task/32476e169ad31810 and sometimes I get a page like this: https://luci-logdog.appspot.com/v/?s=chrome%2Fbb%2Fchromium.perf%2FMac_Pro_10.11_Perf%2F77%2F%2B%2Frecipes%2Fsteps%2Fs__trigger__page_cycler_v2.intl_es_fr_pt-BR_on_ATI_GPU_on_Mac_on_Mac-10.11%2F0%2Fstdout# that leads to this: https://luci-logdog.appspot.com/v/?s=chrome%2Fbb%2Fchromium.perf%2FMac_Pro_10.11_Perf%2F77%2F%2B%2Frecipes%2Fsteps%2Fpage_cycler_v2.intl_es_fr_pt-BR_on_ATI_GPU_on_Mac_on_Mac-10.11%2F0%2Fstdout So the big bug is that the dashboard stdio links are 404s, but I'm not sure what the right link for the user is.
,
Nov 9 2016
FYI, you need to log in to see data on the dashboard. The problem is that the step is named "page_cycler_v2.intl_es_fr_pt-BR on ATI GPU on Mac on Mac-10.11", but the link assumes the name is "page_cycler_v2.intl_es_fr_pt-BR" This is caused by the swarming code, which adds something about ATI GPU etc.. to the step. I've seen this code before, let me find it. I don't like it, because it confuses sheriff-o-matic. I'm fine with removing it, but we should see why it was added in the first place.
,
Nov 9 2016
https://codereview.chromium.org/1720873002 was the change which added this.
,
Nov 9 2016
Assigning to pawel; do we still need the os dimensions in the step name?
,
Nov 9 2016
The reason for adding the OS is that the same test can be triggered multiple time on multiple OSes. The OS name is only present when it's not using the "default" value as per the recipe.
,
Nov 10 2016
+1 to removing this at least for the perf use case Besides the issue with the perf dashboard links (which I am wondering if we can somehow fix in the upload script and strip the step name) this makes it very hard to quickly see what is failing on a given host bot on the perf waterfall. For the perf use case, each host bot will be triggering jobs on swarming bots that all have the same dimensions (os, gpu etc). Therefore, if you want the name of the os you can look at the host bot name. When looking at the waterfall overview page it is hard to determine at a glance what is failing when every failing benchmark as the boilerplate os and version as it is part of the failing step name.
,
Nov 15 2016
My suggestion would be to make dimensions part of step text by default (not step name), and only add them to step names if we need to disambiguate (when same test is triggered with different dimensions on the same bot). Important thing to note is default swarming dimensions may be different from OS running the buildbot slave, so I found it helpful to make effective dimensions clearly visible somewhere in the UI to notice mistakes/mismatches easier. Assigning to Andy for crossover team. Feel free to assign back to me.
,
Nov 15 2016
This bug is not untriaged.
,
Nov 16 2016
ping Who is owning the implementation of this so we can prioritize it and move it forward? I would be happy to help out with implementation with a little guidance.
,
Nov 16 2016
Go for it, Emily. Feel free to reach out on this bug as your progress through the problem. The right group of people is CC’d here.
,
Nov 16 2016
Personally, I think just adding the OS information to the step text would be good enough. Unfortunately, a lot of stuff depends on step names, so I think we should just keep the step name clean and equal to the test name.
,
Dec 1 2016
I'm going to take this.
,
Feb 2 2017
,
Feb 10 2017
,
Mar 15 2018
I think the consensus here was that step names aren't great, but there's not an easy way to fix them? I'm not sure. Ned, should this still be worked on?
,
Mar 15 2018
I am not sure this feature is still useful in the world of single perf setp, where a log may be quite big. I put this blocked on CTU. We can reevaluate this once that project is done.
,
Jan 7
I am closing this given we now have a direct link to "Benchmark_logs" from the build page as well as included links in SOM. |
|||||||||||
►
Sign in to add a comment |
|||||||||||
Comment 1 by mar...@chromium.org
, Nov 9 2016