New issue
Advanced search Search tips

Issue 769623 link

Starred by 1 user

Issue metadata

Status: WontFix
Owner:
Closed: Dec 2017
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: ----
Pri: 2
Type: Task



Sign in to add a comment

Request for relaxing the timeout of bots for perf benchmark

Project Member Reported by shimazu@chromium.org, Sep 28 2017

Issue description

After crrev.com/c/684034, we expect the loading.mobile benchmark will take longer time to run than before. Therefore, we'd like to request relaxing the timeout for the bot to run.

The CL will make the PWA stories in loading.mobile story set to be run in 3 different cache states:
{Cold, Warm, and Hot}. We reduced the number of pages from 10->5, but this will still increase the number of total page loads by 10.

 
Components: -Speed>Benchmarks Speed>Benchmarks>Waterfall
Owner: eyaich@chromium.org
Status: Untriaged (was: Available)
Assign to Emily for triaging
Cc: eyaich@chromium.org
Owner: martiniss@chromium.org
Status: Assigned (was: Untriaged)
We can increase the timeout if we need to. I'm wary to, as loading.mobile is already one of the longer benchmarks, and this will increase cycle time. They already take 4.5 hours (https://cs.chromium.org/chromium/src/tools/perf/core/perf_data_generator.py?q=perf_data_gene&sq=package:chromium&l=816), which is pretty extreme.

I'm cc-ing annie to make sure she's ok with this change. If that's the case, we can bump up the timeout more. 

Do you have any idea how long this benchmark will take once the linked CL lands?

As an aside, there's an architectural reason we can't really handle long running single benchmarks, which we're currently working to get rid of. In the future, we should be able to add more hardware and offset a benchmark taking longer.
We've landed crrev.com/c/684034 and it looks taking 10 minutes more.
However, the test looks succeeding now as long as I'm checking around chromeperf dashboard:
https://chromeperf.appspot.com/report?sid=41b4dc32c443e9db403fb3a59cb2a14755b6b69b8f201e819b018878b860b8e3&rev=505529

Even on Android One, it takes 165-170 min, so 4.5 hours (270 min) seems enough.
I'm not familiar with the bots, so could you do double-checking it and confirm if we need more timeout?

Sorry for confusing! 

Thanks for checking shimazu@, I think we should add extra 15 minutes timeout just to be safe.
Status: WontFix (was: Assigned)
This hasn't seemed necessary. I'm going to tentatively close this for now.

Sign in to add a comment