New issue
Advanced search Search tips

Issue 757067 link

Starred by 2 users

Issue metadata

Status: Fixed
Owner:
Closed: Sep 2017
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: ----
Pri: 2
Type: Bug

Blocking:
issue 731558



Sign in to add a comment

Failure in webkitpy executive_unittest on LUCI linux try bot

Project Member Reported by qyears...@chromium.org, Aug 18 2017

Issue description


[125/1410] webkitpy.common.system.executive_unittest.ExecutiveTest.test_run_command_with_unicode failed unexpectedly:
  Traceback (most recent call last):
    File "/b/swarming/w/ir/cache/builder/linux/src/third_party/WebKit/Tools/Scripts/webkitpy/common/system/executive_unittest.py", line 129, in test_run_command_with_unicode
      output = executive.run_command(command_line('echo', unicode_tor_input))
    File "/b/swarming/w/ir/cache/builder/linux/src/third_party/WebKit/Tools/Scripts/webkitpy/common/system/executive.py", line 296, in run_command
      close_fds=self._should_close_fds())
    File "/b/swarming/w/ir/cache/builder/linux/src/third_party/WebKit/Tools/Scripts/webkitpy/common/system/executive.py", line 364, in popen
      return subprocess.Popen(string_args, **kwargs)
    File "/usr/lib/python2.7/subprocess.py", line 710, in __init__
      errread, errwrite)
    File "/usr/lib/python2.7/subprocess.py", line 1327, in _execute_child
      raise child_exception
  TypeError: execv() arg 2 must contain only strings


Job:
https://ci.chromium.org/swarming/task/380fe94cecb7d110

Log:
https://luci-logdog.appspot.com/v/?s=chromium%2Fbuildbucket%2Fcr-buildbucket.appspot.com%2F8970890543104452432%2F%2B%2Fsteps%2Fwebkit_python_tests__with_patch_%2F0%2Fstdout

 
Components: Infra>Platform
Labels: -Pri-3 Pri-2
I'm hitting this, too, and am using No-Equivalent-Builders because of it.
The relevant section of code is:

https://cs.chromium.org/chromium/src/third_party/WebKit/Tools/Scripts/webkitpy/common/system/executive.py?l=355

Currently we convert all args to the `unicode` type, which generally seems to work, but apparently not on some platforms.

Looking at the log above, this is clearly Python2.7 on Linux. Not sure why this exception is happening here.

My only idea is that instead of converting to `unicode`, we should perhaps convert all args to `str`, encoded as utf8.
Cc: dpranke@chromium.org
+dpranke fyi
Cc: no...@chromium.org
Labels: LUCI-ClosedBeta-Bug
+nodir

We stumbled upon this yesterday I think.
I wonder if there's some locale setting that's different that's affecting things?

We should set up these tests to run under swarming to see if that's part of the issue.
Cc: d...@chromium.org jbudorick@chromium.org qyears...@chromium.org
Owner: dpranke@chromium.org
Status: Started (was: Available)
Very strange. Once I converted the tests to run under swarming, the problem went away. 

This suggests that there's something specific to the LUCI/kitchen environment that is messing things up. Or possibly vpython (??).

I've posted a CL that'll work around (?) the problem accordingly, by running the tests under swarming. Actually using swarming for these tests is a waste, but making it possible to do so flushed out a few interesting things.

Comment 7 by d...@chromium.org, Sep 4 2017

More debugging output (like what the actual list is) would be useful. Both Linux and Mac bundled Python versions seem happy with unicode.

There being a difference between raw Swarming environment and LUCI environment is deliberate; however, none of the differences seem obviously at fault here. It's also possible that the test script is doing something differently in the LUCI environment, causing it to make an invalid request.

To debug this, I think the first useful thing would be to print the subprocess command.

Comment 8 by no...@chromium.org, Sep 5 2017

Blocking: 731558

Comment 9 by no...@chromium.org, Sep 5 2017

Cc: estaab@chromium.org tansell@chromium.org
 Issue 756945  has been merged into this issue.
Cc: -tansell@chromium.org
Project Member

Comment 11 by bugdroid1@chromium.org, Sep 6 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/43a5c81154242b1297c669ab28db4de1fd1ff080

commit 43a5c81154242b1297c669ab28db4de1fd1ff080
Author: Dirk Pranke <dpranke@chromium.org>
Date: Wed Sep 06 03:25:41 2017

Swarm the webkit_python_tests to make LUCI happy.

For reasons that aren't immediately obvious, one of the 
webkitpy unittests seems to fail when run locally as part of a 
LUCI build. This CL converts the tests to run under swarming
to work around it.

Bug:  757067  
Change-Id: I252f6d961894c1a4cfe890bf71295a29ebf3de96
No-Try: true
Reviewed-on: https://chromium-review.googlesource.com/648640
Commit-Queue: Dirk Pranke <dpranke@chromium.org>
Reviewed-by: Quinten Yearsley <qyearsley@chromium.org>
Cr-Commit-Position: refs/heads/master@{#499873}
[modify] https://crrev.com/43a5c81154242b1297c669ab28db4de1fd1ff080/BUILD.gn
[modify] https://crrev.com/43a5c81154242b1297c669ab28db4de1fd1ff080/testing/buildbot/chromium.linux.json
[modify] https://crrev.com/43a5c81154242b1297c669ab28db4de1fd1ff080/testing/buildbot/chromium.mac.json
[modify] https://crrev.com/43a5c81154242b1297c669ab28db4de1fd1ff080/testing/buildbot/chromium.win.json
[modify] https://crrev.com/43a5c81154242b1297c669ab28db4de1fd1ff080/testing/buildbot/gn_isolate_map.pyl
[modify] https://crrev.com/43a5c81154242b1297c669ab28db4de1fd1ff080/third_party/WebKit/Tools/Scripts/test-webkitpy
[modify] https://crrev.com/43a5c81154242b1297c669ab28db4de1fd1ff080/third_party/WebKit/Tools/Scripts/webkitpy/layout_tests/run_webkit_tests_unittest.py
[modify] https://crrev.com/43a5c81154242b1297c669ab28db4de1fd1ff080/third_party/WebKit/Tools/Scripts/webkitpy/style/checkers/python_unittest.py

Status: Fixed (was: Started)
I've filed bug 762634 to follow up on the test I had to disable (webkitpy.style.checkers.python_unittest.StyleChecker.test_check).

I've also filed  bug 762633  to follow up on the unicode issue (why this test is failing in the LUCI recipe environment, but not anywhere else).

Apart from that, this issue is fixed now, I think.

Comment 13 by efoo@chromium.org, Oct 7 2017

Labels: LUCI-Blocker-M4

Comment 14 by efoo@chromium.org, Jan 31 2018

Labels: -LUCI-Blocker-M4 luci-blocker-migration

Comment 15 by efoo@chromium.org, Feb 13 2018

Labels: -LUCI-blocker-migration LUCI-Chromium-CQSets LUCI-Blocker-Chromium-CQSets

Sign in to add a comment