Issue 806136 link

Starred by 2 users

Issue metadata

Status:	WontFix
Owner:	p...@chromium.org
Closed:	Jan 2018
Cc:	mar...@chromium.org
Components:	Test
EstimatedDays:	----
NextAction:	----
OS:	----
Pri:	3
Type:	Bug

Setting the GTEST_FILTER environment variable has no effect on Windows

Project Member Reported by p...@chromium.org, Jan 26 2018

Issue description

What steps will reproduce the problem?
(1) python tools/swarming_client/swarming.py trigger -S https://chromium-swarm.appspot.com -I https://isolateserver.appspot.com -s ff6e7942e2a5e0b2a8ab0afaf3813bb12feab26d -d os Windows-7-SP1 -d pool Chrome -e GTEST_FILTER IEImporterBrowserTest.IE7ImporterPasswordsTest

What is the expected result?
Only the IEImporterBrowserTest.IE7ImporterPasswordsTest test runs.

What happens instead?
All tests run.

Comment 1 by mar...@chromium.org, Jan 26 2018

Cc: mar...@chromium.org
Components: -Infra>Platform>Swarming Test
Owner: p...@chromium.org
Status: Assigned (was: Untriaged)

Peter, please give a link to the task. I ran the command and got this task:
https://chromium-swarm.appspot.com/task?id=3b4c0a3070be8710

The environment variable is correctly set in the task.
https://isolateserver.appspot.com/browse?namespace=default-gzip&hash=ff6e7942e2a5e0b2a8ab0afaf3813bb12feab26d

If browser_tests correctly handles GTEST_FILTER is outside the scope of Swarming. Can you confirm that running this code locally with the environment variable set does what you want? Or said otherwise, what makes you think that the GTEST_FILTER environment variable is supported?

Another issue, you are triggering a 64 bits test but didn't specify the CPU. I changed it to a 10 just to make running faster. Compare:

python tools/swarming_client/swarming.py trigger -S https://chromium-swarm.appspot.com -I https://isolateserver.appspot.com -s ff6e7942e2a5e0b2a8ab0afaf3813bb12feab26d -d os Windows-10 -d cpu x86-64 -d pool Chrome -e GTEST_FILTER IEImporterBrowserTest.IE7ImporterPasswordsTest
-> https://chromium-swarm.appspot.com/user/task/3b4c12d9fd47f510

python tools/swarming_client/swarming.py trigger -S https://chromium-swarm.appspot.com -I https://isolateserver.appspot.com -s ff6e7942e2a5e0b2a8ab0afaf3813bb12feab26d -d os Windows-10 -d cpu x86-64 -d pool Chrome -- --gtest_filter=IEImporterBrowserTest.IE7ImporterPasswordsTest
-> https://chromium-swarm.appspot.com/user/task/3b4c124398d31b10

Comment 2 by p...@chromium.org, Jan 26 2018

Status: WontFix (was: Assigned)

My mistake, I get the same result by setting GTEST_FILTER locally. So it does appear that I was unaware that Google Test does not respect GTEST_FILTER. I somehow assumed that it was a standard feature of Google Test, but taking a closer look it appears to be an optional feature:

https://cs.chromium.org/chromium/src/third_party/googletest/src/googletest/src/gtest.cc?type=cs&q=GTEST_TEST_FILTER_ENV_VAR_&l=194

Also, I didn't know that you could specify additional command line arguments to swarming jobs like that. I don't know how I managed to miss it since it was right at the top of the help message for "swarming.py trigger". Thanks for the hint!