Request for relaxing the timeout of bots for perf benchmark |
|||
Issue descriptionAfter crrev.com/c/684034, we expect the loading.mobile benchmark will take longer time to run than before. Therefore, we'd like to request relaxing the timeout for the bot to run. The CL will make the PWA stories in loading.mobile story set to be run in 3 different cache states: {Cold, Warm, and Hot}. We reduced the number of pages from 10->5, but this will still increase the number of total page loads by 10.
,
Sep 28 2017
,
Sep 28 2017
We can increase the timeout if we need to. I'm wary to, as loading.mobile is already one of the longer benchmarks, and this will increase cycle time. They already take 4.5 hours (https://cs.chromium.org/chromium/src/tools/perf/core/perf_data_generator.py?q=perf_data_gene&sq=package:chromium&l=816), which is pretty extreme. I'm cc-ing annie to make sure she's ok with this change. If that's the case, we can bump up the timeout more. Do you have any idea how long this benchmark will take once the linked CL lands? As an aside, there's an architectural reason we can't really handle long running single benchmarks, which we're currently working to get rid of. In the future, we should be able to add more hardware and offset a benchmark taking longer.
,
Oct 2 2017
We've landed crrev.com/c/684034 and it looks taking 10 minutes more. However, the test looks succeeding now as long as I'm checking around chromeperf dashboard: https://chromeperf.appspot.com/report?sid=41b4dc32c443e9db403fb3a59cb2a14755b6b69b8f201e819b018878b860b8e3&rev=505529 Even on Android One, it takes 165-170 min, so 4.5 hours (270 min) seems enough. I'm not familiar with the bots, so could you do double-checking it and confirm if we need more timeout? Sorry for confusing!
,
Oct 2 2017
Thanks for checking shimazu@, I think we should add extra 15 minutes timeout just to be safe.
,
Dec 28 2017
This hasn't seemed necessary. I'm going to tentatively close this for now. |
|||
►
Sign in to add a comment |
|||
Comment 1 by nedngu...@google.com
, Sep 28 2017Owner: eyaich@chromium.org
Status: Untriaged (was: Available)