New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 836458 link

Starred by 1 user

Issue metadata

Status: Fixed
Owner:
Closed: Apr 2018
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: ----
Pri: 2
Type: Bug
Proj-Servicification

Blocking:
issue 822484



Sign in to add a comment

Mojo Linux Perf: test_pre_run.[error] loading.desktop.network_service '*.isolated file for target telemetry_perf_tests is missing'

Project Member Reported by chongz@chromium.org, Apr 24 2018

Issue description

The bot started to consistently fail since build #5383:
https://ci.chromium.org/buildbot/chromium.perf.fyi/Mojo%20Linux%20Perf/5383

And unfortunately the build contains 618 changes...
+nednguyen@, johnw@: Do we have any recent changes to the bot infra?

Thanks!

(Note that the previous build #5382 also fails but for an unrelated reason)

Sample Log:
```test_pre_run.[error] loading.desktop.network_service on NVIDIA GPU on Linux
python -u /tmp/tmp7Pzpl8.py telemetry_perf_tests
in dir /b/c/b/Mojo_Linux_Perf:
 allow_subannotations: False
 base_name: 'test_pre_run.[error] loading.desktop.network_service on NVIDIA GPU on Linux'
 cmd: ['python', '-u', '/tmp/tmp7Pzpl8.py', 'telemetry_perf_tests']
 cwd: '/b/c/b/Mojo_Linux_Perf'
 env: {'CHROME_HEADLESS': '1'}
 env_prefixes: {'PATH': '/b/cipd_client:/b/cipd_path_tools:/b/cipd_path_tools/bin'}
 env_suffixes: {}
 infra_step: False
 name: 'test_pre_run.[error] loading.desktop.network_service on NVIDIA GPU on Linux'
 nest_level: 1
 ok_ret: frozenset([0])
 trigger_specs: ()
full environment:
 AWS_CREDENTIAL_FILE: /b/build/site_config/.boto
 BOTO_CONFIG: /b/build/site_config/.boto
 BUILDBOT_BLAMELIST: [u'aee@chromium.org', u'afakhry@google.com', ..., u'yuweih@chromium.org', u'zhongyi@chromium.org', u'zmo@chromium.org']
 BUILDBOT_BRANCH: master
 BUILDBOT_BUILDBOTURL: http://build.chromium.org/p/chromium.perf.fyi/
 BUILDBOT_BUILDERNAME: Mojo Linux Perf
 BUILDBOT_BUILDNUMBER: 5383
 BUILDBOT_CLOBBER: 
 BUILDBOT_GOT_REVISION: None
 BUILDBOT_MASTERNAME: chromium.perf.fyi
 BUILDBOT_REVISION: a76b1a9850634705577ac1b4c89a4e3934d6959e
 BUILDBOT_SCHEDULER: None
 BUILDBOT_SLAVENAME: slave146-c1
 CHROME_HEADLESS: 1
 CIPD_CACHE_DIR: /b/c/cipd
 DISPLAY: :0.0
 GIT_USER_AGENT: git/2.11.0 linux2 slave146-c1.c.chromecompute.google.com.internal
 HOME: /home/chrome-bot
 INFRA_BUILDBOT_MASTER_CLASS_NAME: ChromiumPerfFyi
 INFRA_BUILDBOT_SLAVE_ACTIVE_SUBDIR: 
 INFRA_BUILDBOT_SLAVE_NAME: slave146-c1
 LANG: en_US.UTF-8
 LOGDOG_COORDINATOR_HOST: logs.chromium.org
 LOGDOG_STREAM_PREFIX: bb/chromium.perf.fyi/Mojo_Linux_Perf/5383
 LOGDOG_STREAM_PROJECT: chrome
 LOGDOG_STREAM_SERVER_PATH: unix:/b/build/rr/tmp8EHMw4/butler.sock
 PAGER: cat
 PATH: /b/cipd_client:/b/cipd_path_tools:/b/cipd_path_tools/bin:/b/cipd_path_tools:/b/cipd_path_tools/bin:/b/cipd_client:/home/chrome-bot/slavebin:/b/depot_tools:/usr/bin:/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/bin
 PWD: /b/build/slave/Mojo_Linux_Perf/build
 PYTHONIOENCODING: UTF-8
 PYTHONPATH: /b/build/site_config:/b/build/scripts:/b/build/scripts/release:/b/build/third_party:/b/build/third_party/google_api_python_client:/b/build/third_party/httplib2/python2:/b/build/third_party/infra_libs:/b/build/third_party/oauth2client:/b/build/third_party/pyasn1:/b/build/third_party/pyasn1-modules:/b/build/third_party/python-rsa:/b/build/third_party/requests_2_10_0:/b/build/third_party/setuptools-0.6c11:/b/build/third_party/site-packages:/b/build/third_party/uritemplate:/b/build_internal/site_config:/b/build_internal/symsrc:/b/build/slave:/b/build/third_party/buildbot_slave_8_4:/b/build/third_party/twisted_10_2:
 PYTHONUNBUFFERED: 1
 TESTING_SLAVENAME: slave146-c1
 USER: chrome-bot
 USERNAME: chrome-bot
 VPYTHON_CLEAR_PYTHONPATH: 1
 VPYTHON_VIRTUALENV_ROOT: /b/c/vpython
*.isolated file for target telemetry_perf_tests is missing
step returned non-zero exit code: 1
```

 
Cc: eyaich@chromium.org
Emily may know what's going on 

Comment 2 by eyaich@chromium.org, Apr 25 2018

Cc: eakuefner@chromium.org simonhatch@chromium.org
So this started failing when we migrated our linux perf bots on the main waterfall to our new isolate target: performance_test_suite

I am not 100% sure how the builder decides what isolates to build, but I am assuming it loads the associated chromium.*.json file and builds all the targets that are present there for the builders it triggers.

Since Mojo Linux Perf and Histogram Pipeline Linux Perf both use the Linux Builder Perf on the main waterfall: https://uberchromegw.corp.google.com/i/chromium.perf/builders/Linux%20Builder%20Perf/builds/5743 maybe it can't see the json cross waterfalls?  I don't have insight into builders on one master and testers that it triggers on other ones.  But you can see that it is not being built: https://logs.chromium.org/v/?s=chrome%2Fbb%2Fchromium.perf%2FLinux_Builder_Perf%2F5743%2F%2B%2Frecipes%2Fsteps%2Fgenerate_build_files%2F0%2Flogs%2Fswarming-targets-file.txt%2F0


I am assuming if we switch them to using the linux builder on the fyi waterfall it will start picking up the correct json in the chromium.perf.fyi.json file and build the right targets.

Adding Ethan and Simon as an FYI on the histogram bot.

Project Member

Comment 3 by bugdroid1@chromium.org, Apr 25 2018

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/tools/build/+/80940a89cc82f08cca98eb220d9c4b39a6000451

commit 80940a89cc82f08cca98eb220d9c4b39a6000451
Author: Emily Hanley <eyaich@google.com>
Date: Wed Apr 25 18:38:18 2018

Updating perf fyi linux testers to use fyi builder

Had to add no coverage to api for chromium_tests as there doesn't appear
to be anything else exercising the cross master builders.

Bug:836458
Change-Id: Ifb520d65df850c2a53484b8bd584262d0a6918d6
Reviewed-on: https://chromium-review.googlesource.com/1028131
Reviewed-by: Ned Nguyen <nednguyen@google.com>
Commit-Queue: Emily Hanley <eyaich@chromium.org>

[modify] https://crrev.com/80940a89cc82f08cca98eb220d9c4b39a6000451/scripts/slave/recipe_modules/chromium_tests/chromium_perf_fyi.py
[modify] https://crrev.com/80940a89cc82f08cca98eb220d9c4b39a6000451/scripts/slave/recipe_modules/chromium_tests/api.py
[modify] https://crrev.com/80940a89cc82f08cca98eb220d9c4b39a6000451/scripts/slave/recipe_modules/chromium_tests/tests/api/archive_build.expected/linux_builder.json

Comment 4 by eyaich@chromium.org, Apr 26 2018

Owner: eyaich@chromium.org
Status: Fixed (was: Untriaged)

Sign in to add a comment