New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.

Issue 743615 link

Starred by 1 user

Issue metadata

Status: Fixed
Owner:
Please use my @vewd.com account ins...
Closed: Jul 2017
Cc:
EstimatedDays: ----
NextAction: ----
OS: Linux , Windows , Chrome , Mac
Pri: 3
Type: Bug



Sign in to add a comment

174.5% regression in browser_tests at 486768:486779

Project Member Reported by niklase@chromium.org, Jul 15 2017

Issue description

Comment 1 by most...@opera.com, Jul 15 2017

Status: Started (was: Available)
My CL (https://chromium-review.googlesource.com/c/569159/) changed the default number of parallel test jobs, based on the command line flags being used- if a specific number is specified on the command line use that, otherwise if running a subset of the tests and not running in bot mode.  Where can I find the command line for browser_tests as used here?

For reference, here is the original code before I started touching this (this block is now extracted into a separate function, but the processing order should be the same):
https://chromium.googlesource.com/chromium/src/+/6614832e3a90564cb4e24c02eea867a9666b3ca0/base/test/launcher/test_launcher.cc#806

The list of performance changes over that range is small (2), and WebRTC is the only regression, and only on windows (but I don't know what the platform coverage is like):
https://chromeperf.appspot.com/group_report?rev=486779

If this is due to these WebRTC tests being much more sensitive to using > 1 parallel test threads, perhaps they should be moved to a separate unittest binary that forces the use of only one parallel test job?
mostynb, you can find some details about how these tests are run in https://bugs.chromium.org/p/chromium/issues/detail?id=736837

Comment 3 by most...@opera.com, Jul 16 2017

Ah right.  This is a silly error that I should have spotted.  Fix here:
https://chromium-review.googlesource.com/c/573380/

Once again, sorry for the disruption.
Project Member

Comment 4 by bugdroid1@chromium.org, Jul 16 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/eb12aa53060d9397de49ed5c2e72a0bdf36833e2

commit eb12aa53060d9397de49ed5c2e72a0bdf36833e2
Author: Mostyn Bramley-Moore <mostynb@opera.com>
Date: Sun Jul 16 17:45:42 2017

don't forget to actually return the number of specified test jobs

BUG= 741927 , 743615 
TEST=Running browser_tests with the flags below, should print "Using 1 parallel jobs."
--run-manual --ui-test-action-max-timeout=350000 --test-launcher-jobs=1 --test-launcher-bot-mode --test-launcher-print-test-stdio=always
TBR=phajdan.jr@chromium.org

Change-Id: I33f2b4fc54b43ebec8a27c532da17146b42c07cd
Reviewed-on: https://chromium-review.googlesource.com/573380
Reviewed-by: Mostyn Bramley-Moore <mostynb@opera.com>
Commit-Queue: Mostyn Bramley-Moore <mostynb@opera.com>
Cr-Commit-Position: refs/heads/master@{#487014}
[modify] https://crrev.com/eb12aa53060d9397de49ed5c2e72a0bdf36833e2/base/test/launcher/test_launcher.cc

Comment 5 by most...@opera.com, Jul 16 2017

Status: Fixed (was: Started)
Still waiting for the perf graphs, but this ongoing build on the waterfall says that it's using "1 parallel jobs" for browser_tests again:
https://build.chromium.org/p/chromium.webrtc/builders/Win10%20Tester/builds/17926/steps/browser_tests/logs/stdio

Sign in to add a comment