New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 699547 link

Starred by 1 user

Issue metadata

Status: Fixed
Owner:
Last visit > 30 days ago
Closed: Mar 2017
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: Android
Pri: 2
Type: Bug



Sign in to add a comment

Test failures caused by unable to find depot tools on Lollipop Phone Tester.

Project Member Reported by jbudorick@chromium.org, Mar 8 2017

Issue description

https://codereview.chromium.org/2701473003/ added google_storage_helper.py, which uses find_depot_tools. It's currently causing failures like this on https://luci-milo.appspot.com/buildbot/chromium.android/Lollipop%20Phone%20Tester/ (at least):

Failed to find depot_tools
Traceback (most recent call last):
  File "/b/s/w/irWysYvV/build/android/test_runner.py", line 35, in <module>
    from pylib.base import test_run_factory
  File "/b/s/w/irWysYvV/build/android/pylib/base/test_run_factory.py", line 12, in <module>
    from pylib.local.device import local_device_instrumentation_test_run
  File "/b/s/w/irWysYvV/build/android/pylib/local/device/local_device_instrumentation_test_run.py", line 23, in <module>
    from pylib.utils import google_storage_helper
  File "/b/s/w/irWysYvV/build/android/pylib/utils/google_storage_helper.py", line 22, in <module>
    import find_depot_tools  # pylint: disable=import-error
  File "/b/s/w/irWysYvV/build/find_depot_tools.py", line 49, in <module>
    import breakpad
ImportError: No module named breakpad
 
Cc: jbudorick@chromium.org
waaaat. Is this something weird about how swarming bots are configured. find_depot_tools is needed by "gclient runhooks" so it should work fine on all of our bots... I assumed.
I'm surprised that it's working on *some* of the swarming bots rather than all or none.
Cc: bpastene@chromium.org
According to Ben, all the bots are supposed to have depot_tools at /b/depot_tools.

chrome-bot@build134-b1:(Linux 14.04):/b$ ls
s

The bot that ran these tests doesnt seem to have depot_tools. Not sure what is up.
Summary: Test failures caused by unable to find depot tools on Lollipop Phone Tester. (was: Addition of pylib/utils/google_storage_helper.py caused several tests on Lollipop Phone Tester to fail)
Yeah, having a depot_tools checkout under /b/ is kind of an unwritten dependency for a lot of buildbot-ish things. (It gets there by being declared as a slave DEPs at https://chrome-internal.googlesource.com/chrome/tools/build/slave.DEPS.git/+/master/DEPS which our machine setup script syncs with)

But this is a swarming bot, and everything a swarming test needs *should* be packaged up into the isolate. That said, it's probably easier to just manually fetch it on the bot in the usual place for now.

And FTR, the depot_tools dependency took me by surprise too. I had to add it to my docker containers at rf0baf7444a2b95f2b43c24ae7a12365aa6e20568. Must be relatively new.
swarming can also fetch CIPD packages. Is there one for depot_tools?
Cc: vadimsh@chromium.org iannucci@chromium.org
Don't think so. +some people. Do we package depot_tools in CIPD anywhere?

I know we zip-ify it at https://pantheon.corp.google.com/storage/browser/chrome-infra/ but IIUC that's mainly for Windows bots.


Cc: d...@chromium.org
swarming bots intentionally don't provide an unpinned version of depot_tools (unlike buildbot). Recipes, however, do have a pinned copy of depot_tools that you can use (use the "depot_tools/depot_tools": https://chromium.googlesource.com/chromium/tools/depot_tools/+/master/recipe_modules/depot_tools/api.py).

'find_depot_tools.py' needs to be rooted out and destroyed everywhere. What's the script using it for?
Yeah it's only using it to find gsutil.py: https://cs.chromium.org/chromium/src/build/android/pylib/utils/google_storage_helper.py?type=cs&q=google_storage%2B+package:%5Echromium$&l=39

We should pass the path to gsutil.py in as an argument.
gsutil is also checked into places like catapult/third_party/. I wonder if we can just access the copy that lives there (since our test runner depends pretty heavily on catapult code anyways). Might be cleaner than passing it as an arg.
that would also work.

at some point in the future (hopefully sooner than later), recipes should have the ability to put pinned binary dependencies on $PATH via cipd, but that's not there yet.
Reverted the change where I added the find_depot_tools import to test runner.
https://codereview.chromium.org/2737223002/

Hopefully that fixes the bot. Will reland the change without find_depot_tools.
Status: Fixed (was: Assigned)

Sign in to add a comment