elm-paladin becomes the slowest slave |
|||||||||
Issue descriptioncollected a CQ summary for the past month, elm-paladin was the slowest slave for 62 times. This slows the CQ runs and needs investigation. 16:41:32: INFO: Slowest CQ slaves out of 93 passing runs: 16:41:32: INFO: 6 times the slowest slave was daisy_spring-paladin 16:41:32: INFO: 50th percentile: 1.50 hours, 90th percentile: 1.66 hours 16:41:32: INFO: 11 times the slowest slave was wolf-tot-paladin 16:41:32: INFO: 50th percentile: 1.47 hours, 90th percentile: 2.26 hours 16:41:32: INFO: 62 times the slowest slave was elm-paladin 16:41:32: INFO: 50th percentile: 2.07 hours, 90th percentile: 2.65 hours
,
Oct 3 2016
Some side-by-side comparisons between wolf-paladin https://viceroy.corp.google.com/chromeos/build_details?build_config=wolf-paladin&build_number=12321 and elm-paladin https://viceroy.corp.google.com/chromeos/build_details?build_id=1091877 Some of the main differences: BuildPackages elm: 32m wolf: 22m BuildImage elm: 15m wolf: 8m HWTest elm: 40m wolf: 31m
,
Oct 12 2016
,
Oct 18 2016
By randomly picking some paladins and comparing the results, the elm-paladin takes longer time to finish BuildPackages and BuildImage. The elm-paladin buildPackages runs more packages than the other paladins. Among the ~661 packages in the elm-paladin, chromeos-fonts-0.0.1-r24 usually takes the longest time to compile. If elm-paladin has to build all the packages, we're fine with it taking more time; if some packages are not necessary, we should consult the right owners to remove the unneeded packages from the dependency.
,
Nov 5 2016
Aseda, mind taking an initial stab at this to understand why Elm paladins are consistently slow? Packages wise Elm should be very similar to Minnie since they both support ARC++. Is it ARM64 vs 32 that is making a difference here?
,
Nov 5 2016
Sure, I'll try to get to it next week. The ARM64 vs 32 might be worth looking into.
,
Nov 12 2016
Didn't get to it this week, maybe next week.
,
Nov 25 2016
Comparing gru and elm (both arm64 wtih ARC++): https://viceroy.corp.google.com/chromeos/build_details?build_config=elm-paladin&build_number=1177 https://viceroy.corp.google.com/chromeos/build_details?build_config=gru-paladin&build_number=1212 36m vs 20m https://viceroy.corp.google.com/chromeos/build_details?build_config=elm-paladin&build_number=1176 https://viceroy.corp.google.com/chromeos/build_details?build_config=gru-paladin&build_number=1211 32m vs 17m https://viceroy.corp.google.com/chromeos/build_details?build_config=elm-paladin&build_number=1175 https://viceroy.corp.google.com/chromeos/build_details?build_config=gru-paladin&build_number=1210 31m vs 17m cat elm-1177.html | sed -n 's/Completed \(.*\) (in \([0-9]*\)m\([0-9\.]*\)s)/\1;\2;\3/p' | sort > elm-1177.txt cat gru-1212.html | sed -n 's/Completed \(.*\) (in \([0-9]*\)m\([0-9\.]*\)s)/\1;\2;\3/p' | sort > gru-1212.txt Playing a bit with these numbers (ruby script attached): https://docs.google.com/spreadsheets/d/14AHmQh21J48DAJ7a3goG9SvNLBsfoPhe5CHq-wvK1oI/edit#gid=0&vpid=A598 Lots of packages are much slower to build on elm vs gru, e.g. all the autotest packages: chromeos-base/autotest-tests-0.0.4-r7853 gru: 105.9s elm: 356.7s chromeos-base/telemetry-0.0.1-r6 gru: 122.7s elm: 309.8s but then some are much faster... chromeos-base/autotest-deps-clipboard-0.4-r1 gru:32.2s elm:8.7s
,
Feb 13 2017
,
Apr 17 2017
,
May 30 2017
,
Aug 1 2017
,
Oct 14 2017
|
|||||||||
►
Sign in to add a comment |
|||||||||
Comment 1 Deleted