New issue
Advanced search Search tips
Note: Color blocks (like █ or █) mean that a user may not be available. Tooltip shows the reason.

Issue 708370 link

Starred by 1 user
Status: WontFix
Owner: ----
Closed: Apr 2017
Cc:
█ phajdan.jr@chromium.org
█ phajdan@google.com
dpranke@chromium.org
Components:
EstimatedDays: ----
NextAction: ----
OS: ----
Pri: 3
Type: Bug



Sign in to add a comment

gtest results in groups of 10

Project Member Reported by cblume@chromium.org, Apr 5 2017 
I am adding a test and running it locally. When I restrict the test with --gtest_filter="*mytestgroup*" it seems to clump the first 10 tests together and put the remaining 2 tests in a separate group.

When I run without --gtest_filter my tests are once again separated, but in a different way.




$ out/Debug/blink_platform_unittests --gtest_filter="*SegmentStream*"
IMPORTANT DEBUGGING NOTE: batches of tests are run inside their
own process. For debugging a test inside a debugger, use the
--gtest_filter=<your_test_name> flag along with
--single-process-tests.
Using sharding settings from environment. This is shard 0/1
Using 1 parallel jobs.
Note: Google Test filter = SegmentStreamTest.defaultContructor_clearsPosition:SegmentStreamTest.defaultConstructor_setsIsAtEnd:SegmentStreamTest.defaultConstructor_setsIsCleared:SegmentStreamTest.setReader_unsetsIsAtEnd:SegmentStreamTest.setReader_unsetsIsCleared:SegmentStreamTest.setReader_setsIsAtEnd_whenSetToNull:SegmentStreamTest.setReader_setsIsCleared_whenSetToNull:SegmentStreamTest.setReader_doesNotChangePosition_whenSetToNull:SegmentStreamTest.read_consumesBuffer:SegmentStreamTest.read_doesNotConsumeBuffer_whenCleared
[==========] Running 10 tests from 1 test case.
[----------] Global test environment set-up.
[----------] 10 tests from SegmentStreamTest
[ RUN      ] SegmentStreamTest.defaultContructor_clearsPosition
[       OK ] SegmentStreamTest.defaultContructor_clearsPosition (1 ms)
[ RUN      ] SegmentStreamTest.defaultConstructor_setsIsAtEnd
[       OK ] SegmentStreamTest.defaultConstructor_setsIsAtEnd (0 ms)
[ RUN      ] SegmentStreamTest.defaultConstructor_setsIsCleared
[       OK ] SegmentStreamTest.defaultConstructor_setsIsCleared (0 ms)
[ RUN      ] SegmentStreamTest.setReader_unsetsIsAtEnd
[       OK ] SegmentStreamTest.setReader_unsetsIsAtEnd (0 ms)
[ RUN      ] SegmentStreamTest.setReader_unsetsIsCleared
[       OK ] SegmentStreamTest.setReader_unsetsIsCleared (0 ms)
[ RUN      ] SegmentStreamTest.setReader_setsIsAtEnd_whenSetToNull
[       OK ] SegmentStreamTest.setReader_setsIsAtEnd_whenSetToNull (0 ms)
[ RUN      ] SegmentStreamTest.setReader_setsIsCleared_whenSetToNull
[       OK ] SegmentStreamTest.setReader_setsIsCleared_whenSetToNull (0 ms)
[ RUN      ] SegmentStreamTest.setReader_doesNotChangePosition_whenSetToNull
[       OK ] SegmentStreamTest.setReader_doesNotChangePosition_whenSetToNull (0 ms)
[ RUN      ] SegmentStreamTest.read_consumesBuffer
[       OK ] SegmentStreamTest.read_consumesBuffer (1 ms)
[ RUN      ] SegmentStreamTest.read_doesNotConsumeBuffer_whenCleared
[       OK ] SegmentStreamTest.read_doesNotConsumeBuffer_whenCleared (0 ms)
[----------] 10 tests from SegmentStreamTest (2 ms total)

[----------] Global test environment tear-down
[==========] 10 tests from 1 test case ran. (2 ms total)
[  PASSED  ] 10 tests.
[1/12] SegmentStreamTest.defaultContructor_clearsPosition (1 ms)
[2/12] SegmentStreamTest.defaultConstructor_setsIsAtEnd (0 ms)
[3/12] SegmentStreamTest.defaultConstructor_setsIsCleared (0 ms)
[4/12] SegmentStreamTest.setReader_unsetsIsAtEnd (0 ms)
[5/12] SegmentStreamTest.setReader_unsetsIsCleared (0 ms)
[6/12] SegmentStreamTest.setReader_setsIsAtEnd_whenSetToNull (0 ms)
[7/12] SegmentStreamTest.setReader_setsIsCleared_whenSetToNull (0 ms)
[8/12] SegmentStreamTest.setReader_doesNotChangePosition_whenSetToNull (0 ms)
[9/12] SegmentStreamTest.read_consumesBuffer (1 ms)
[10/12] SegmentStreamTest.read_doesNotConsumeBuffer_whenCleared (0 ms)
Note: Google Test filter = SegmentStreamTest.read_updatesPosition:SegmentStreamTest.peek_doesNotConsumeBuffer_whenCleared
[==========] Running 2 tests from 1 test case.
[----------] Global test environment set-up.
[----------] 2 tests from SegmentStreamTest
[ RUN      ] SegmentStreamTest.read_updatesPosition
[       OK ] SegmentStreamTest.read_updatesPosition (0 ms)
[ RUN      ] SegmentStreamTest.peek_doesNotConsumeBuffer_whenCleared
[       OK ] SegmentStreamTest.peek_doesNotConsumeBuffer_whenCleared (0 ms)
[----------] 2 tests from SegmentStreamTest (0 ms total)

[----------] Global test environment tear-down
[==========] 2 tests from 1 test case ran. (1 ms total)
[  PASSED  ] 2 tests.
[11/12] SegmentStreamTest.read_updatesPosition (0 ms)
[12/12] SegmentStreamTest.peek_doesNotConsumeBuffer_whenCleared (0 ms)
SUCCESS: all tests passed.
Tests took 0 seconds.








$ out/Debug/blink_platform_unittests
...
[599/1712] SegmentStreamTest.defaultContructor_clearsPosition (0 ms
[600/1712] SegmentStreamTest.defaultConstructor_setsIsAtEnd (0 ms
[601/1712] PaintArtifactCompositorTestWithPropertyTrees.OneTransform (4 ms)
[602/1712] PaintArtifactCompositorTestWithPropertyTrees.TransformCombining (2 ms
[603/1712] PaintArtifactCompositorTestWithPropertyTrees.FlattensInheritedTransform (3 ms)
[604/1712] PaintArtifactCompositorTestWithPropertyTrees.SortingContextID (2 ms
[605/1712] PaintArtifactCompositorTestWithPropertyTrees.OneClip (2 ms
[606/1712] PaintArtifactCompositorTestWithPropertyTrees.NestedClips (3 ms)
[607/1712] PaintArtifactCompositorTestWithPropertyTrees.DeeplyNestedClips (2 ms
[608/1712] PaintArtifactCompositorTestWithPropertyTrees.SiblingClips (2 ms)
[609/1712] PaintArtifactCompositorTestWithPropertyTrees.ForeignLayerPassesThrough (1 ms)
[610/1712] PaintArtifactCompositorTestWithPropertyTrees.EffectTreeConversion (2 ms)
[611/1712] PaintArtifactCompositorTestWithPropertyTrees.OneScrollNode (3 ms)
[612/1712] PaintArtifactCompositorTestWithPropertyTrees.TransformUnderScrollNode (2 ms)
[613/1712] PaintArtifactCompositorTestWithPropertyTrees.NestedScrollNodes (2 ms
[614/1712] PaintArtifactCompositorTestWithPropertyTrees.MergeSimpleChunks (2 ms
[615/1712] PaintArtifactCompositorTestWithPropertyTrees.MergeClip (2 ms)
[616/1712] PaintArtifactCompositorTestWithPropertyTrees.Merge2DTransform (2 ms
[617/1712] PaintArtifactCompositorTestWithPropertyTrees.Merge2DTransformDirectAncestor (3 ms)
[618/1712] PaintArtifactCompositorTestWithPropertyTrees.MergeTransformOrigin (3 ms)
[619/1712] PaintArtifactCompositorTestWithPropertyTrees.MergeOpacity (3 ms)
[620/1712] PaintArtifactCompositorTestWithPropertyTrees.MergeNested (2 ms)
[621/1712] PaintArtifactCompositorTestWithPropertyTrees.ClipPushedUp (3 ms)
[622/1712] PaintArtifactCompositorTestWithPropertyTrees.ClipAndEffectNoTransform (3 ms)
[623/1712] PaintArtifactCompositorTestWithPropertyTrees.TwoClips (2 ms
[624/1712] PaintArtifactCompositorTestWithPropertyTrees.TwoTransformsClipBetween (2 ms)
[625/1712] PaintArtifactCompositorTestWithPropertyTrees.OverlapTransform (3 ms
[626/1712] PaintArtifactCompositorTestWithPropertyTrees.MightOverlap (1 ms)
[627/1712] PaintArtifactCompositorTestWithPropertyTrees.PendingLayer (1 ms)
[628/1712] PaintArtifactCompositorTestWithPropertyTrees.PendingLayerWithGeometry (1 ms)
[629/1712] PaintArtifactCompositorTestWithPropertyTrees.TransformWithElementId (2 ms)
[630/1712] PaintArtifactCompositorTestWithPropertyTrees.EffectWithElementId (2 ms
[631/1712] SegmentStreamTest.defaultConstructor_setsIsCleared (0 ms
[632/1712] SegmentStreamTest.setReader_unsetsIsAtEnd (0 ms)
[633/1712] SegmentStreamTest.setReader_unsetsIsCleared (0 ms)
[634/1712] SegmentStreamTest.setReader_setsIsAtEnd_whenSetToNull (0 ms
[635/1712] SegmentStreamTest.setReader_setsIsCleared_whenSetToNull (1 ms)
[636/1712] SegmentStreamTest.setReader_doesNotChangePosition_whenSetToNull (0 ms
[637/1712] SegmentStreamTest.read_consumesBuffer (0 ms)
[638/1712] SegmentStreamTest.read_doesNotConsumeBuffer_whenCleared (0 ms)
[639/1712] SegmentStreamTest.read_updatesPosition (0 ms)
[640/1712] SegmentStreamTest.peek_doesNotConsumeBuffer_whenCleared (0 ms)
...

Comment 1 by phajdan.jr@chromium.org, Apr 27 2017

Cc: phajdan@google.com
Status: WontFix (was: Untriaged)
This is working as expected, unless I'm missing something.

Comment 2 by dpranke@chromium.org, Apr 27 2017 

@phajdan.jr - can you explain why things are being clumped that way, i.e., what the test launcher is doing?

Comment 3 by cblume@google.com, Apr 27 2017 

An update from my side:
I have since added more tests and it clumps into groups of 10.
My initial report here only mentioned the first group of 10 being clumped.

But it is clearly setting up, running 10 tests, tearing down, then repeating for the next 10 tests until it finishes them all.


I am curious why this behavior was chosen. But only curious. It does not adversely effect me other than maybe running setup/teardown more then required and thus taking slightly longer in the aggregate.

Comment 4 by phajdan.jr@chromium.org, Jun 6 2017 

The test launcher runs tests in separate processes to parallelize and gracefully handle hungs and crashes. For fast unit tests the overhead of separate processes is so large that we have to batch tests to make the overhead negligible. On developer workstations we don't parallelize by default, to make the output more readable and resembling vanilla gtest, but still batch so it doesn't differ too much from production environment.

Feel free to ask further questions (although not sure if bugs are the best place for that).
►

Sign in to add a comment