New issue
Advanced search Search tips
Note: Color blocks (like █ or █) mean that a user may not be available. Tooltip shows the reason.

Issue 726212 link

Starred by 1 user
Status: Fixed
Owner:
█ rnep...@chromium.org
Last visit > 30 days ago
Closed: May 2017
Cc:
█ mgiuca@chromium.org
ehmaldonado@chromium.org
sullivan@chromium.org
Components:
EstimatedDays: ----
NextAction: ----
OS: ----
Pri: 1
Type: Bug



Sign in to add a comment

"benchmarks.benchmark_smoke_unittest.BenchmarkSmokeTest.webrtc" is flaky

Project Member Reported by chromium...@appspot.gserviceaccount.com, May 25 2017 
"benchmarks.benchmark_smoke_unittest.BenchmarkSmokeTest.webrtc" is flaky.

This issue was created automatically by the chromium-try-flakes app. Please find the right owner to fix the respective test/step and assign this issue to them. If the step/test is infrastructure-related, please add Infra-Troopers label and change issue status to Untriaged. When done, please remove the issue from Sheriff Bug Queue by removing the Sheriff-Chromium label.

We have detected 6 recent flakes. List of all flakes can be found at https://chromium-try-flakes.appspot.com/all_flake_occurrences?key=ahVzfmNocm9taXVtLXRyeS1mbGFrZXNySAsSBUZsYWtlIj1iZW5jaG1hcmtzLmJlbmNobWFya19zbW9rZV91bml0dGVzdC5CZW5jaG1hcmtTbW9rZVRlc3Qud2VicnRjDA.

Flaky tests should be disabled within 30 minutes unless culprit CL is found and reverted. Please see more details here: https://sites.google.com/a/chromium.org/dev/developers/tree-sheriffs/sheriffing-bug-queues#triaging-auto-filed-flakiness-bugs

Comment 1 by mgiuca@chromium.org, May 25 2017

Cc: sullivan@chromium.org mgiuca@chromium.org
Labels: -Sheriff-Chromium
Owner: ehmaldonado@chromium.org
Status: Assigned (was: Untriaged)
I think this is caused by tools/perf/benchmarks/webrtc.py.

I tried to disable WebrtcPerfBenchmark using @benchmark.Disabled. But I got a gigantic presubmit error. Plus I don't know if this is the right thing to disable (BenchmarkSmokeTest says to disable the file in t/p/benchmarks).

Sorry, I'm just a sheriff and don't understand telemetry tests, they are different to normal Chrome tests. Assigning to Edward to disable or deal with. Thanks.

Comment 2 by ehmaldonado@chromium.org, May 25 2017

Cc: ehmaldonado@chromium.org
Owner: rnep...@chromium.org
rnephew@: Can you take a look, please?

Comment 3 by rnep...@chromium.org, May 25 2017

Status: Fixed (was: Assigned)
This is already fixed.


This was the error:
Traceback (most recent call last):
    RunBenchmark at e:\b\s\w\ir\third_party\catapult\telemetry\telemetry\internal\story_runner.py:389
      benchmark.ValidateExpectations(stories)
    ValidateExpectations at e:\b\s\w\ir\third_party\catapult\telemetry\telemetry\benchmark.py:273
      self._expectations.ValidateAgainstStorySet(story_set)
    ValidateAgainstStorySet at e:\b\s\w\ir\third_party\catapult\telemetry\telemetry\story\expectations.py:31
      raise TypeError('Story %s is not in the story set.' % expectation)
  TypeError: Story multiple_peerconnections is not in the story set.
►

Sign in to add a comment