New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 702201 link

Starred by 1 user

Issue metadata

Status: Duplicate
Merged: issue 663802
Owner:
Closed: Mar 2017
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: Android
Pri: ----
Type: ----



Sign in to add a comment

webrtc.webrtc_smoothness_tbmv2 failing on Android

Project Member Reported by sullivan@chromium.org, Mar 16 2017

Issue description

webrtc.webrtc_smoothness_tbmv2 failing on 4 builders

Builders failed on: 
- Android Nexus5 Perf (1): 
  https://build.chromium.org/p/chromium.perf/builders/Android%20Nexus5%20Perf%20%281%29
- Android Nexus6 Perf (1): 
  https://build.chromium.org/p/chromium.perf/builders/Android%20Nexus6%20Perf%20%281%29
- Android Nexus7v2 Perf (1): 
  https://build.chromium.org/p/chromium.perf/builders/Android%20Nexus7v2%20Perf%20%281%29
- Android One Perf (1): 
  https://build.chromium.org/p/chromium.perf/builders/Android%20One%20Perf%20%281%29

Looks like this has been failing since it was added. I'm going to disable while a fix is worked on. 

Sample log:
https://luci-logdog.appspot.com/v/?s=chrome%2Fbb%2Fchromium.perf%2FAndroid_Nexus6_Perf__1_%2F5031%2F%2B%2Frecipes%2Fsteps%2Fwebrtc.webrtc_smoothness_tbmv2%2F0%2Fstdout
[ RUN      ] canvas_capture_peer_connection
...
Traceback (most recent call last):
  File "/b/c/b/Android_Nexus6_Perf__1_/src/third_party/catapult/telemetry/telemetry/internal/story_runner.py", line 88, in _RunStoryAndProcessErrorIfNeeded
    state.RunStory(results)
  File "/b/c/b/Android_Nexus6_Perf__1_/src/third_party/catapult/common/py_trace_event/py_trace_event/trace_event_impl/decorators.py", line 75, in traced_function
    return func(*args, **kwargs)
  File "/b/c/b/Android_Nexus6_Perf__1_/src/third_party/catapult/telemetry/telemetry/page/shared_page_state.py", line 299, in RunStory
    self._current_page.Run(self)
  File "/b/c/b/Android_Nexus6_Perf__1_/src/third_party/catapult/telemetry/telemetry/page/__init__.py", line 112, in Run
    self.RunPageInteractions(action_runner)
  File "/b/c/b/Android_Nexus6_Perf__1_/src/tools/perf/page_sets/webrtc_cases.py", line 152, in RunPageInteractions
    action_runner.ExecuteJavaScript('draw();')
  File "/b/c/b/Android_Nexus6_Perf__1_/src/third_party/catapult/common/py_trace_event/py_trace_event/trace_event_impl/decorators.py", line 75, in traced_function
    return func(*args, **kwargs)
  File "/b/c/b/Android_Nexus6_Perf__1_/src/third_party/catapult/telemetry/telemetry/internal/actions/action_runner.py", line 214, in ExecuteJavaScript
    return self._tab.ExecuteJavaScript(*args, **kwargs)
  File "/b/c/b/Android_Nexus6_Perf__1_/src/third_party/catapult/common/py_trace_event/py_trace_event/trace_event_impl/decorators.py", line 75, in traced_function
    return func(*args, **kwargs)
  File "/b/c/b/Android_Nexus6_Perf__1_/src/third_party/catapult/telemetry/telemetry/internal/browser/web_contents.py", line 144, in ExecuteJavaScript
    return self._inspector_backend.ExecuteJavaScript(*args, **kwargs)
  File "/b/c/b/Android_Nexus6_Perf__1_/src/third_party/catapult/common/py_trace_event/py_trace_event/trace_event_impl/decorators.py", line 75, in traced_function
    return func(*args, **kwargs)
  File "/b/c/b/Android_Nexus6_Perf__1_/src/third_party/catapult/telemetry/telemetry/internal/backends/chrome_inspector/inspector_backend.py", line 37, in inner
    return func(inspector_backend, *args, **kwargs)
  File "/b/c/b/Android_Nexus6_Perf__1_/src/third_party/catapult/telemetry/telemetry/internal/backends/chrome_inspector/inspector_backend.py", line 216, in ExecuteJavaScript
    self._runtime.Execute(statement, context_id, timeout)
  File "/b/c/b/Android_Nexus6_Perf__1_/src/third_party/catapult/telemetry/telemetry/internal/backends/chrome_inspector/inspector_runtime.py", line 21, in Execute
    self.Evaluate(expr + '; 0;', context_id, timeout)
  File "/b/c/b/Android_Nexus6_Perf__1_/src/third_party/catapult/telemetry/telemetry/internal/backends/chrome_inspector/inspector_runtime.py", line 54, in Evaluate
    description=details.get('exception', {}).get('description'))
EvaluateException: UncaughtReferenceError:
ReferenceError: draw is not defined
    at <anonymous>:1:1

 
Cc: nednguyen@chromium.org
Do you know what might be the problem?
Components: Speed>Benchmarks
File "/b/c/b/Android_Nexus6_Perf__1_/src/tools/perf/page_sets/webrtc_cases.py", line 152, in RunPageInteractions
    action_runner.ExecuteJavaScript('draw();')
...
"ReferenceError: draw is not defined".

This means due to the way you set up the page, the "draw" method still isn't defined when you run "draw();". I suspect this is because this page is slow on mobile, so you probably need to wait for some time before you can invoke "    action_runner.ExecuteJavaScript('draw();')"

Project Member

Comment 3 by bugdroid1@chromium.org, Mar 16 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/9c7221f573308689650a423da0e02744c8b1d02a

commit 9c7221f573308689650a423da0e02744c8b1d02a
Author: sullivan <sullivan@chromium.org>
Date: Thu Mar 16 15:28:40 2017

Disable failing webrtc.webrtc_smoothness_tbmv2 benchmark.

Also remove stray print.

BUG= 702201 
TBR=nednguyen@google.com

Review-Url: https://codereview.chromium.org/2752043003
Cr-Commit-Position: refs/heads/master@{#457441}

[modify] https://crrev.com/9c7221f573308689650a423da0e02744c8b1d02a/tools/perf/benchmarks/webrtc.py

Mergedinto: 663802
Status: Duplicate (was: Available)
Why are you reverting the CL? The fix was right, but after fixing it, you got a different problem. So the next step should be fixing the 2nd problem, not undo the fix to the 1st problem.
Reverting to disable.
But you're right, I should just add the disable decorator.
Project Member

Comment 10 by bugdroid1@chromium.org, Mar 28 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/78eee2ed541a59974558681e9165a18a0fe5028b

commit 78eee2ed541a59974558681e9165a18a0fe5028b
Author: ehmaldonado <ehmaldonado@chromium.org>
Date: Tue Mar 28 13:16:54 2017

Use local pages for webrtc telemetry tests.

This is to avoid hitting HTTPS errors.

BUG= chromium:702201 
NOTRY=True

Review-Url: https://codereview.chromium.org/2761163003
Cr-Commit-Position: refs/heads/master@{#460078}

[modify] https://crrev.com/78eee2ed541a59974558681e9165a18a0fe5028b/PRESUBMIT.py
[delete] https://crrev.com/0990801e0a56421e48fda78935487f8c1e670931/tools/perf/page_sets/data/webrtc_audio_cases.json
[delete] https://crrev.com/0990801e0a56421e48fda78935487f8c1e670931/tools/perf/page_sets/data/webrtc_audio_cases_001.wpr.sha1
[delete] https://crrev.com/0990801e0a56421e48fda78935487f8c1e670931/tools/perf/page_sets/data/webrtc_datachannel_cases.json
[delete] https://crrev.com/0990801e0a56421e48fda78935487f8c1e670931/tools/perf/page_sets/data/webrtc_datachannel_cases_001.wpr.sha1
[delete] https://crrev.com/0990801e0a56421e48fda78935487f8c1e670931/tools/perf/page_sets/data/webrtc_getusermedia_cases.json
[delete] https://crrev.com/0990801e0a56421e48fda78935487f8c1e670931/tools/perf/page_sets/data/webrtc_getusermedia_cases_001.wpr.sha1
[delete] https://crrev.com/0990801e0a56421e48fda78935487f8c1e670931/tools/perf/page_sets/data/webrtc_peerconnection_cases.json
[delete] https://crrev.com/0990801e0a56421e48fda78935487f8c1e670931/tools/perf/page_sets/data/webrtc_peerconnection_cases_003.wpr.sha1
[delete] https://crrev.com/0990801e0a56421e48fda78935487f8c1e670931/tools/perf/page_sets/data/webrtc_smoothness_cases.json
[delete] https://crrev.com/0990801e0a56421e48fda78935487f8c1e670931/tools/perf/page_sets/data/webrtc_smoothness_cases_001.wpr.sha1
[delete] https://crrev.com/0990801e0a56421e48fda78935487f8c1e670931/tools/perf/page_sets/data/webrtc_stresstest_cases.json
[delete] https://crrev.com/0990801e0a56421e48fda78935487f8c1e670931/tools/perf/page_sets/data/webrtc_stresstest_cases_000.wpr.sha1
[delete] https://crrev.com/0990801e0a56421e48fda78935487f8c1e670931/tools/perf/page_sets/data/webrtc_stresstest_cases_001.wpr.sha1
[add] https://crrev.com/78eee2ed541a59974558681e9165a18a0fe5028b/tools/perf/page_sets/update_webrtc_cases
[modify] https://crrev.com/78eee2ed541a59974558681e9165a18a0fe5028b/tools/perf/page_sets/webrtc_cases.py
[add] https://crrev.com/78eee2ed541a59974558681e9165a18a0fe5028b/tools/perf/page_sets/webrtc_cases/adapter.js
[add] https://crrev.com/78eee2ed541a59974558681e9165a18a0fe5028b/tools/perf/page_sets/webrtc_cases/audio.html
[add] https://crrev.com/78eee2ed541a59974558681e9165a18a0fe5028b/tools/perf/page_sets/webrtc_cases/audio.js
[add] https://crrev.com/78eee2ed541a59974558681e9165a18a0fe5028b/tools/perf/page_sets/webrtc_cases/canvas-capture.html
[add] https://crrev.com/78eee2ed541a59974558681e9165a18a0fe5028b/tools/perf/page_sets/webrtc_cases/canvas-capture.js
[add] https://crrev.com/78eee2ed541a59974558681e9165a18a0fe5028b/tools/perf/page_sets/webrtc_cases/common.js
[add] https://crrev.com/78eee2ed541a59974558681e9165a18a0fe5028b/tools/perf/page_sets/webrtc_cases/constraints.html
[add] https://crrev.com/78eee2ed541a59974558681e9165a18a0fe5028b/tools/perf/page_sets/webrtc_cases/constraints.js
[add] https://crrev.com/78eee2ed541a59974558681e9165a18a0fe5028b/tools/perf/page_sets/webrtc_cases/datatransfer.html
[add] https://crrev.com/78eee2ed541a59974558681e9165a18a0fe5028b/tools/perf/page_sets/webrtc_cases/datatransfer.js
[add] https://crrev.com/78eee2ed541a59974558681e9165a18a0fe5028b/tools/perf/page_sets/webrtc_cases/multiple-peerconnections.html
[add] https://crrev.com/78eee2ed541a59974558681e9165a18a0fe5028b/tools/perf/page_sets/webrtc_cases/multiple-peerconnections.js
[add] https://crrev.com/78eee2ed541a59974558681e9165a18a0fe5028b/tools/perf/page_sets/webrtc_cases/resolution.html
[add] https://crrev.com/78eee2ed541a59974558681e9165a18a0fe5028b/tools/perf/page_sets/webrtc_cases/resolution.js

Project Member

Comment 11 by bugdroid1@chromium.org, Mar 29 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/f244469aa781823cdde22de772e1ca1944f55b2f

commit f244469aa781823cdde22de772e1ca1944f55b2f
Author: ehmaldonado <ehmaldonado@chromium.org>
Date: Wed Mar 29 06:33:46 2017

Enable WebRTC telemetry tests on Android.

BUG= chromium:702201 

Review-Url: https://codereview.chromium.org/2778233002
Cr-Commit-Position: refs/heads/master@{#460301}

[modify] https://crrev.com/f244469aa781823cdde22de772e1ca1944f55b2f/tools/perf/benchmarks/webrtc.py

Sign in to add a comment