The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/tools/build/+/e096fcfd77f8e45fb52190c716ecb39821b89ac6

commit e096fcfd77f8e45fb52190c716ecb39821b89ac6
Author: Yuzhu Shen <yzshen@chromium.org>
Date: Thu May 11 17:01:25 2017

Add a linux FYI bot for running perf tests with the new network service.

Bug:717738
Change-Id: I86c99d05dccc13919b2ba90312fe5ed3f92280fe
Reviewed-on: https://chromium-review.googlesource.com/502091
Reviewed-by: Dirk Pranke <dpranke@chromium.org>
Reviewed-by: Vadim Shtayura <vadimsh@chromium.org>
Commit-Queue: Yuzhu Shen <yzshen@chromium.org>

[add] https://crrev.com/e096fcfd77f8e45fb52190c716ecb39821b89ac6/scripts/slave/recipes/chromium.expected/full_chromium_perf_fyi_Mojo_Linux_Perf.json
[modify] https://crrev.com/e096fcfd77f8e45fb52190c716ecb39821b89ac6/scripts/slave/recipes/chromium.expected/full_chromium_perf_Linux_Builder.json
[modify] https://crrev.com/e096fcfd77f8e45fb52190c716ecb39821b89ac6/masters/master.chromium.perf.fyi/slaves.cfg
[modify] https://crrev.com/e096fcfd77f8e45fb52190c716ecb39821b89ac6/scripts/slave/recipe_modules/chromium_tests/chromium_perf_fyi.py
[modify] https://crrev.com/e096fcfd77f8e45fb52190c716ecb39821b89ac6/masters/master.chromium.perf.fyi/master.cfg
[modify] https://crrev.com/e096fcfd77f8e45fb52190c716ecb39821b89ac6/scripts/slave/recipe_modules/chromium_tests/tests/api/archive_build.expected/linux_builder.json

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/5bab1b8217a0045f0c949ba3ae6ffc864e3b8732

commit 5bab1b8217a0045f0c949ba3ae6ffc864e3b8732
Author: Yuzhu Shen <yzshen@chromium.org>
Date: Thu May 11 19:47:46 2017

Network services: add tools/perf/contrib/network_service.

The folder will be used to host benchmarks for measuring perf of Chrome with
the network service enabled.

Eventually we won't need this folder when the network service is enabled by
default and the old network-in-browser-process code path is removed.

BUG= 717738 ,598073
R=nednguyen@google.com

Review-Url: https://codereview.chromium.org/2877553003 .
Cr-Commit-Position: refs/heads/master@{#471048}

[add] https://crrev.com/5bab1b8217a0045f0c949ba3ae6ffc864e3b8732/tools/perf/contrib/network_service/OWNERS

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/987b0d546a3b0cc49512a8f2b2fdfe1996c6f0db

commit 987b0d546a3b0cc49512a8f2b2fdfe1996c6f0db
Author: yzshen <yzshen@chromium.org>
Date: Fri Jul 07 19:26:33 2017

Enable loading.desktop benchmark with network service enabled on perf fyi bot.

BUG= 717738 

Review-Url: https://codereview.chromium.org/2973733002
Cr-Commit-Position: refs/heads/master@{#485007}

[modify] https://crrev.com/987b0d546a3b0cc49512a8f2b2fdfe1996c6f0db/testing/buildbot/chromium.perf.fyi.json
[add] https://crrev.com/987b0d546a3b0cc49512a8f2b2fdfe1996c6f0db/tools/perf/chromium.perf.fyi.extras.json
[add] https://crrev.com/987b0d546a3b0cc49512a8f2b2fdfe1996c6f0db/tools/perf/contrib/network_service/__init__.py
[add] https://crrev.com/987b0d546a3b0cc49512a8f2b2fdfe1996c6f0db/tools/perf/contrib/network_service/loading.py
[modify] https://crrev.com/987b0d546a3b0cc49512a8f2b2fdfe1996c6f0db/tools/perf/core/perf_data_generator.py
[modify] https://crrev.com/987b0d546a3b0cc49512a8f2b2fdfe1996c6f0db/tools/perf/core/perf_data_generator_unittest.py

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/tools/build/+/9ff1bca7885b47aea98e7d7f1a5cc415482fa2ea

commit 9ff1bca7885b47aea98e7d7f1a5cc415482fa2ea
Author: Yuzhu Shen <yzshen@chromium.org>
Date: Fri Jul 07 21:06:04 2017

Change Mojo Linux Perf to run tests listed in chromium.perf.fyi.json.

BUG= 717738 

Change-Id: I12f7491500ede888bf8fc8c7a326ed341e21c310
Reviewed-on: https://chromium-review.googlesource.com/563779
Commit-Queue: Yuzhu Shen <yzshen@chromium.org>
Reviewed-by: Stephen Martinis <martiniss@chromium.org>

[modify] https://crrev.com/9ff1bca7885b47aea98e7d7f1a5cc415482fa2ea/scripts/slave/recipe_modules/chromium_tests/chromium_perf_fyi.py

The task is running on the bot now (https://chromium-swarm.appspot.com/task?id=375155f8b754e310&refresh=10&show_raw=1 is an example), but failing with this error: 


libdc1394 error: Failed to initialize libdc1394
INFO:devil.utils.cmd_helper:[host]> /b/s/w/ir/third_party/catapult/devil/bin/deps/linux2/x86_64/bin/adb devices
INFO:devil.utils.cmd_helper:[host]> /b/s/w/ir/third_party/catapult/devil/bin/deps/linux2/x86_64/bin/adb devices
INFO:devil.utils.cmd_helper:[host]> /b/s/w/ir/third_party/catapult/devil/bin/deps/linux2/x86_64/bin/adb kill-server
WARNING:root:Chrome build location for linux_x86_64 not found. Browser will be run without Flash.
WARNING:root:Found (release), but you do not have a DISPLAY environment set.
INFO:devil.utils.cmd_helper:[host]> /b/s/w/ir/third_party/catapult/devil/bin/deps/linux2/x86_64/bin/adb devices
INFO:devil.utils.cmd_helper:[host]> /b/s/w/ir/third_party/catapult/devil/bin/deps/linux2/x86_64/bin/adb devices
INFO:devil.utils.cmd_helper:[host]> /b/s/w/ir/third_party/catapult/devil/bin/deps/linux2/x86_64/bin/adb kill-server
WARNING:root:Chrome build location for linux_x86_64 not found. Browser will be run without Flash.
WARNING:root:Found (release), but you do not have a DISPLAY environment set.

You aren't running on any particular bot, so I have a feeling that running on arbitrary GCE machines is causing you issues. I'm not that familiar with how perf bots differ from random GCE machines though.

Hmm... I am guessing maybe it is caused by some missing environment variable, based on this log:
"WARNING:root:Found (release), but you do not have a DISPLAY environment set."
I could try to figure out how to add this variable, but if you know the right way to do it please let me know.

If we need to run the test on a specific bot, is it sufficient for me to change the config in tools/perf/chromium.perf.fyi.extra.json to get rid of the "swarming" section?

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/fd7957902206f5958800cbed6203ba8b9a01dd81

commit fd7957902206f5958800cbed6203ba8b9a01dd81
Author: Yuzhu Shen <yzshen@chromium.org>
Date: Fri Jul 28 03:47:48 2017

Attempt to fix tests on  Mojo Linux Perf FYI bot.

The issue is that the test is complaining:
"WARNING:root:Found (release), but you do not have a DISPLAY environment
set."

BUG= 717738 

Change-Id: Ife4228a86fa055416ec20a8049085bf4c2c33ce0
Reviewed-on: https://chromium-review.googlesource.com/590500
Reviewed-by: Dirk Pranke <dpranke@chromium.org>
Commit-Queue: Yuzhu Shen <yzshen@chromium.org>
Cr-Commit-Position: refs/heads/master@{#490249}
[modify] https://crrev.com/fd7957902206f5958800cbed6203ba8b9a01dd81/testing/buildbot/chromium.perf.fyi.json
[modify] https://crrev.com/fd7957902206f5958800cbed6203ba8b9a01dd81/tools/perf/chromium.perf.fyi.extras.json

Unfortunately the CL above doesn't work, it fails with:
------------------------------------------------
Traceback (most recent call last):
  File "../../testing/scripts/run_telemetry_benchmark_as_googletest.py", line 155, in <module>
    sys.exit(main())
  File "../../testing/scripts/run_telemetry_benchmark_as_googletest.py", line 59, in main
    if args.xvfb and xvfb.should_start_xvfb(env):
AttributeError: 'module' object has no attribute 'should_start_xvfb'
------------------------------------------------

I looked at git log and |should_start_xvfb| has been removed quite a while ago in this CL:
https://chromium.googlesource.com/chromium/src/+/8c5f70304b15375ea5b80ccb61617b595abfd287%5E%21/#F1

Dirk and Tom: Would you please advise how to fix this? Thanks!

Hmhh, weird. That means current gpu tests & Telemetry tests are not relying on xvfb at all.

I just check a perf bot job & it's true that we don't use xvfb here:
https://chromium-swarm.appspot.com/task?id=37a0f7b70b163510&refresh=10&show_raw=1

That CL moved the responsibility of should_start_xvfb into run_executable.  I don't think there were any functional changes

As for how to fix it, ptal at the changes to testing/scripts/run_telemetry_as_googletest.py from https://chromium.googlesource.com/chromium/src/+/8c5f70304b15375ea5b80ccb61617b595abfd287%5E%21/#F1

You'll just need to copy the changes over to run_telemetry_benchmark_as_googletest.py

Thanks for reply, Ned and Tom!

Ned: the current telemetry tests have the env variable 'DISPLAY': ':0' set (please see the link in #12); while the swarming task started by the new Mojo Perf FYI bot doesn't have it (please see the link in #8).

One thing that may worth mentioning: it seems the Mojo Perf FYI bot itself has DISPLAY set, e.g., I could find DISPLAY in the bot output:
https://luci-logdog.appspot.com/v/?s=chrome%2Fbb%2Fchromium.perf.fyi%2FMojo_Linux_Perf%2F1745%2F%2B%2Frecipes%2Fsteps%2Floading.desktop.network_service%2F0%2Fstdout

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/0fdf0209666431bed97aa2707ab38117053fbd42

commit 0fdf0209666431bed97aa2707ab38117053fbd42
Author: Yuzhu Shen <yzshen@chromium.org>
Date: Tue Aug 01 20:14:01 2017

Update run_telemetry_benchmark_as_googletest to support --xvfb flag.

This CL follows how run_telemetry_as_googletest.py handles --xvfb flag.

BUG= 717738 

Change-Id: I0bb746d18be7818c782b7086ca96605a60af0761
Reviewed-on: https://chromium-review.googlesource.com/595227
Commit-Queue: Yuzhu Shen <yzshen@chromium.org>
Reviewed-by: Dirk Pranke <dpranke@chromium.org>
Reviewed-by: Thomas Anderson <thomasanderson@chromium.org>
Cr-Commit-Position: refs/heads/master@{#491092}
[modify] https://crrev.com/0fdf0209666431bed97aa2707ab38117053fbd42/testing/scripts/run_telemetry_benchmark_as_googletest.py

Apologies, applied the wrong component in bulk.