New issue
Advanced search Search tips

Issue 880246 link

Starred by 1 user

Issue metadata

Status: Fixed
Owner:
Closed: Sep 11
Components:
EstimatedDays: ----
NextAction: ----
OS: Android
Pri: 1
Type: Bug



Sign in to add a comment

blink run_webkit_tests.py fails with ascii encoding error

Project Member Reported by mattcary@chromium.org, Sep 4

Issue description

This is breaking webkit_layout_tests on, eg, clank KitKat

https://chromium-swarm.appspot.com/task?id=3fbbffff6f4eda10&refresh=10&show_raw=1&wide_logs=true

Example stack trace: 
UnicodeDecodeError raised: 'ascii' codec can't decode byte 0xc3 in position 220382: ordinal not in range(128)
Traceback (most recent call last):
  File "/b/swarming/w/ir/third_party/blink/tools/blinkpy/web_tests/run_webkit_tests.py", line 66, in main
    return run(port, options, args, stderr, stdout).exit_code
  File "/b/swarming/w/ir/third_party/blink/tools/blinkpy/web_tests/run_webkit_tests.py", line 592, in run
    run_details = _run_tests(port, options, args, printer)
  File "/b/swarming/w/ir/third_party/blink/tools/blinkpy/web_tests/run_webkit_tests.py", line 583, in _run_tests
    return manager.run(args)
  File "/b/swarming/w/ir/third_party/blink/tools/blinkpy/web_tests/controllers/manager.py", line 162, in run
    run_results = self._run_test_once(tests_to_run, tests_to_skip, should_retry_failures)
  File "/b/swarming/w/ir/third_party/blink/tools/blinkpy/web_tests/controllers/manager.py", line 241, in _run_test_once
    num_workers)
  File "/b/swarming/w/ir/third_party/blink/tools/blinkpy/web_tests/controllers/manager.py", line 404, in _run_tests
    tests_to_skip, num_workers, retry_attempt)
  File "/b/swarming/w/ir/third_party/blink/tools/blinkpy/web_tests/controllers/layout_test_runner.py", line 121, in run_tests
    pool.run(('test_list', shard.name, shard.test_inputs) for shard in all_shards)
  File "/b/swarming/w/ir/third_party/blink/tools/blinkpy/common/message_pool.py", line 96, in run
    self.wait()
  File "/b/swarming/w/ir/third_party/blink/tools/blinkpy/common/message_pool.py", line 125, in wait
    self._workers[0].run()
  File "/b/swarming/w/ir/third_party/blink/tools/blinkpy/common/message_pool.py", line 269, in run
    self._raise(sys.exc_info())
  File "/b/swarming/w/ir/third_party/blink/tools/blinkpy/common/message_pool.py", line 257, in run
    worker.handle(message.name, message.src, *message.args)
  File "/b/swarming/w/ir/third_party/blink/tools/blinkpy/web_tests/controllers/layout_test_runner.py", line 272, in handle
    device_failed = self._run_test(test_input, test_list_name)
  File "/b/swarming/w/ir/third_party/blink/tools/blinkpy/web_tests/controllers/layout_test_runner.py", line 311, in _run_test
    stop_when_done)
  File "/b/swarming/w/ir/third_party/blink/tools/blinkpy/web_tests/controllers/single_test_runner.py", line 53, in run_single_test
    return runner.run()
  File "/b/swarming/w/ir/third_party/blink/tools/blinkpy/web_tests/controllers/single_test_runner.py", line 135, in run
    return self._run_compare_test()
  File "/b/swarming/w/ir/third_party/blink/tools/blinkpy/web_tests/controllers/single_test_runner.py", line 153, in _run_compare_test
    driver_output = self._driver.run_test(self._driver_input(), self._stop_when_done)
  File "/b/swarming/w/ir/third_party/blink/tools/blinkpy/web_tests/port/driver.py", line 198, in run_test
    self.error_from_test, crash_log, crash_site = self._get_crash_log(text, self.error_from_test, newer_than=start_time)
  File "/b/swarming/w/ir/third_party/blink/tools/blinkpy/web_tests/port/android.py", line 796, in _get_crash_log
    stack)
UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 220382: ordinal not in range(128)
 
Project Member

Comment 1 by bugdroid1@chromium.org, Sep 7

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/caf1e695161420d7f8fee40b8139ccd0244be6a9

commit caf1e695161420d7f8fee40b8139ccd0244be6a9
Author: Matthew Cary <mattcary@chromium.org>
Date: Fri Sep 07 08:38:03 2018

webkit_layout_test: Correctly deal with unicode on stacks

webkit_layout_tests will fail if there is a non-ascii character in the stack
dump. This CL encodes the stack dump to avoid that. Very probably there is some
other problem that is causing the stack to contain non-ascii characters, but at
this point we can't even tell what's the problem due to the crash (see the bug
for details).

Bug:  880246 
Change-Id: I34c5582cc2d4bd98fb17053336e9084624df52a5
Reviewed-on: https://chromium-review.googlesource.com/1204015
Reviewed-by: Dirk Pranke <dpranke@chromium.org>
Commit-Queue: Matthew Cary <mattcary@chromium.org>
Cr-Commit-Position: refs/heads/master@{#589464}
[modify] https://crrev.com/caf1e695161420d7f8fee40b8139ccd0244be6a9/third_party/blink/tools/blinkpy/web_tests/port/android.py

This looks to have just shifted the problem. Looks like stack == None here. I think this is causing this fail running webkit_tests any time a test crashes: https://ci.chromium.org/buildbot/chromium.webkit/WebKit%20Android%20%28Nexus4%29/.

AttributeError raised: 'NoneType' object has no attribute 'encode'
Traceback (most recent call last):
  File "/b/c/b/WebKit_Android__Nexus4_/src/third_party/blink/tools/blinkpy/web_tests/run_webkit_tests.py", line 66, in main
    return run(port, options, args, stderr, stdout).exit_code
  File "/b/c/b/WebKit_Android__Nexus4_/src/third_party/blink/tools/blinkpy/web_tests/run_webkit_tests.py", line 592, in run
    run_details = _run_tests(port, options, args, printer)
  File "/b/c/b/WebKit_Android__Nexus4_/src/third_party/blink/tools/blinkpy/web_tests/run_webkit_tests.py", line 583, in _run_tests
    return manager.run(args)
  File "/b/c/b/WebKit_Android__Nexus4_/src/third_party/blink/tools/blinkpy/web_tests/controllers/manager.py", line 162, in run
    run_results = self._run_test_once(tests_to_run, tests_to_skip, should_retry_failures)
  File "/b/c/b/WebKit_Android__Nexus4_/src/third_party/blink/tools/blinkpy/web_tests/controllers/manager.py", line 241, in _run_test_once
    num_workers)
  File "/b/c/b/WebKit_Android__Nexus4_/src/third_party/blink/tools/blinkpy/web_tests/controllers/manager.py", line 404, in _run_tests
    tests_to_skip, num_workers, retry_attempt)
  File "/b/c/b/WebKit_Android__Nexus4_/src/third_party/blink/tools/blinkpy/web_tests/controllers/layout_test_runner.py", line 121, in run_tests
    pool.run(('test_list', shard.name, shard.test_inputs) for shard in all_shards)
  File "/b/c/b/WebKit_Android__Nexus4_/src/third_party/blink/tools/blinkpy/common/message_pool.py", line 96, in run
    self.wait()
  File "/b/c/b/WebKit_Android__Nexus4_/src/third_party/blink/tools/blinkpy/common/message_pool.py", line 125, in wait
    self._workers[0].run()
  File "/b/c/b/WebKit_Android__Nexus4_/src/third_party/blink/tools/blinkpy/common/message_pool.py", line 269, in run
    self._raise(sys.exc_info())
  File "/b/c/b/WebKit_Android__Nexus4_/src/third_party/blink/tools/blinkpy/common/message_pool.py", line 257, in run
    worker.handle(message.name, message.src, *message.args)
  File "/b/c/b/WebKit_Android__Nexus4_/src/third_party/blink/tools/blinkpy/web_tests/controllers/layout_test_runner.py", line 272, in handle
    device_failed = self._run_test(test_input, test_list_name)
  File "/b/c/b/WebKit_Android__Nexus4_/src/third_party/blink/tools/blinkpy/web_tests/controllers/layout_test_runner.py", line 311, in _run_test
    stop_when_done)
  File "/b/c/b/WebKit_Android__Nexus4_/src/third_party/blink/tools/blinkpy/web_tests/controllers/single_test_runner.py", line 53, in run_single_test
    return runner.run()
  File "/b/c/b/WebKit_Android__Nexus4_/src/third_party/blink/tools/blinkpy/web_tests/controllers/single_test_runner.py", line 135, in run
    return self._run_compare_test()
  File "/b/c/b/WebKit_Android__Nexus4_/src/third_party/blink/tools/blinkpy/web_tests/controllers/single_test_runner.py", line 153, in _run_compare_test
    driver_output = self._driver.run_test(self._driver_input(), self._stop_when_done)
  File "/b/c/b/WebKit_Android__Nexus4_/src/third_party/blink/tools/blinkpy/web_tests/port/driver.py", line 198, in run_test
    self.error_from_test, crash_log, crash_site = self._get_crash_log(text, self.error_from_test, newer_than=start_time)
  File "/b/c/b/WebKit_Android__Nexus4_/src/third_party/blink/tools/blinkpy/web_tests/port/android.py", line 796, in _get_crash_log
    stack.encode('ascii', 'replace'))
AttributeError: 'NoneType' object has no attribute 'encode'

e.g. see https://logs.chromium.org/v/?s=chromium%2Fbb%2Fchromium.webkit%2FWebKit_Android__Nexus4_%2F82357%2F%2B%2Frecipes%2Fsteps%2Fwebkit_tests%2F0%2Fstdout
Could we at least put a try/catch around this code and log an error so that the test suite finishes running? 
At least the tests got farther this time :) I'll make a patch.
Labels: Sheriff-Chromium
CCing Sheriff queue to make aware that this is being worked on.
Thanks!

If you want me to TBR the CL or otherwise expedite its commit, let me know.
Project Member

Comment 7 by bugdroid1@chromium.org, Sep 11

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/9cdf0cc69280095e3a420a1b8d9552656529deda

commit 9cdf0cc69280095e3a420a1b8d9552656529deda
Author: Matthew Cary <mattcary@chromium.org>
Date: Tue Sep 11 10:51:35 2018

webkit_layout_test: Harden stack encoding

The stack returned from _get_stack_from_dump can be None. This hardens
against this case, and hopefully another other unforseen issues around
the stack encoding, by transforming the stack to ascii inside a try
block.

Bug:  880246 
Change-Id: I498b5929dab811a2ccc5e69bb51b930c8547179c
TBR: dpranke@chromium.org
Reviewed-on: https://chromium-review.googlesource.com/1218842
Commit-Queue: Dominic Battré <battre@chromium.org>
Reviewed-by: Dominic Battré <battre@chromium.org>
Cr-Commit-Position: refs/heads/master@{#590258}
[modify] https://crrev.com/9cdf0cc69280095e3a420a1b8d9552656529deda/third_party/blink/tools/blinkpy/web_tests/port/android.py

Status: Fixed (was: Started)

Sign in to add a comment