New issue
Advanced search Search tips

Issue 699771 link

Starred by 1 user

Issue metadata

Status: WontFix
Owner: ----
Closed: Jul 2017
Cc:
EstimatedDays: ----
NextAction: ----
OS: ----
Pri: 2
Type: Bug-Regression

Blocked on:
issue 683184



Sign in to add a comment

3.8%-5.7% regression in page_cycler_v2.intl_hi_ru at 453906:453979

Project Member Reported by benhenry@google.com, Mar 8 2017

Issue description

See the link to graphs below.
 

=== BISECT JOB RESULTS ===
NO Perf regression found

Bisect Details
  Configuration: winx64nvidia_perf_bisect
  Benchmark    : page_cycler_v2.intl_hi_ru
  Metric       : timeToFirstContentfulPaint_avg/pcv1-cold/http___yandex.ru_yandsearch?lr_102567_text_%D0%9F%D0%BE%D0%B3%D0%BE%D0%B4%D0%B0

Revision             Result                  N
chromium@453905      402.223 +- 450.255      21      good
chromium@453979      364.912 +- 88.1736      21      bad

To Run This Test
  src/tools/perf/run_benchmark -v --browser=release_x64 --output-format=chartjson --upload-results --pageset-repeat=1 --also-run-disabled-tests --story-filter=http...yandex.ru.yandsearch.lr.102567.text..D0.9F.D0.BE.D0.B3.D0.BE.D0.B4.D0.B0 page_cycler_v2.intl_hi_ru

Debug Info
  https://chromeperf.appspot.com/buildbucket_job_status/8985647987600934640

Is this bisect wrong?
  https://chromeperf.appspot.com/bad_bisect?try_job_id=6345867105140736


| O O | Visit http://www.chromium.org/developers/speed-infra/perf-bug-faq
|  X  | for more information addressing perf regression bugs. For feedback,
| / \ | file a bug with component Speed>Bisection.  Thank you!
Cc: simonhatch@chromium.org
Simon - compare samples log shows nothing...is that what went wrong here?
Blockedon: 683184
{
  "result": {
    "U": 191,
    "p": 0.46568579971233737,
    "significance": "FAIL_TO_REJECT"
  },
  "sampleA": [
    689.141999989748,
    365.6950000077486,
    358.011000007391,
    358.169999986887,
    357.97100000083447,
    364.32799999415874,
    616.0330000072718,
    356.0460000038147,
    360.386000007391,
    594.3249999880791,
    353.49599999189377,
    354.61200000345707,
    358.23499999940395,
    357.19699999690056,
    465.4350000023842,
    356.9379999935627,
    357.4390000104904,
    353.4270000010729,
    356.22499999403954,
    356.63700000941753,
    356.9259999990463
  ],
  "sampleB": [
    396.4399999976158,
    353.9560000002384,
    359.04999999701977,
    354.7300000041723,
    354.71099999547005,
    356.276000007987,
    388.7769999951124,
    357.0189999938011,
    354.5109999924898,
    434.72699999809265,
    355.6990000009537,
    368.9519999921322,
    355.0300000011921,
    358.3009999990463,
    373.47100000083447,
    358.3969999998808,
    358.36099998652935,
    353.29500000178814,
    355.30799999833107,
    360.0139999985695,
    356.1169999986887
  ]
}

Hmm I'm seeing quite a bit in Compare Samples. From the looks of that, looks like the "bad" value happens very infrequently and the test we use doesn't have the confidence to say for sure a regression happened.

Blocking on crbug.com/683184 for the bisect not proceeding for these types of infrequent outlier regressions.
Project Member

Comment 6 by sheriffbot@chromium.org, Jul 25 2017

Labels: Hotlist-Google
Status: WontFix (was: Untriaged)
This alert was found before M-60 branched. Closing as WontFix as this is believed to either be invalid or non-reproducible. 

Sign in to add a comment